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This interim report presents'progress since the start (December 1970) of ,a 
contract to evaluate the Emergency School Assistance Program. , Since no f-esults 
are yit available, this report Simply describes the study objectives, research ' 
'design, field survey activities, and analysis plans for this study. " - 
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INTRODUCTION 

fil broad terms, Emergency School Assistance Program (ESAP) is a federal program 
to aid locai educational agencies (LEAs)^ in bringing about racial d<^segregation of 
thejr public school systems. 

HBTOBY AND PURPOSE OF ESAP ' * 

The history of ESAP goes back to March 24, 1970, when Presidei|t Nixon issUed 
a statement on school desegregation and highlighted four special categories of. need. ^ , 
In a message to Congress on Maj/21, 1970, the President proposed the Emergency . 
School Aid Act of 1970, to meet thh^ie special needs.. In this new legislation he pro- 
posed l^ee cat^ories of aid t 



• ■ * 

1. Throughout ^his rteport the terms local education agaicy (LEA) and school 
districf^ are* used synonymously. 

"2. (1) The special needs of desegregating (or recently desegregated) 

districts for additional facilities, personnel and training required 
to get the new unitaTy system successfully started; 

. i (2) the special needs of racially impacted schools whdre de facto 
. segregation persists—and where immediate infusions of 
money can make a real difference in terms of educational - 
effectiveness; ■ \ 
. • ' • ■ ' ^ • '• 

1' , (3)' the special needs of those districts tl^at have the furthest 

< to go to catch up educationally with tlie rj^st of tthe nation; and- • 

^ " -. • ' • ' ... • -.• ■ ■. ■• 

(4) the fmancing of innovative techniques'-for providing educationally 
sound interracial experiences for children in racially iscjafced 
schools. ' ' . " 
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(1) aid to districts rfow* eliminating de 
pursuant to direct Federal coui^t 
with plans approved by the. Seer 




'egation either 
in accordance 



of Health, Education, 
and Welfare, for special needs incident to compliance; 

^. (2) aid to districts thiat wifeh to undertake voluntary efforts to 

, eliminate, reduce, or fcrevent de facto racial isolation, with 
such aid specifically targeted for those purposes; and 

^ \Z) aid to districts in which de facto racial s^aration persists, 
foi* the purpose of jielping establish special interratial or 
intercultural educational programs or, where such programs * 
are.impracticable, programs designed to overcome the edu- 
cational disadvantages that stem from raciial isolation. 

He requested one-half billion dollars be appropriated in FY 71 and one billion dollars 
y\pi> 72 for this 'aid.- Anticipating that this proppsed legislation would not he ' 
bnacted in time to be available v/henschoolp op^ed in September 1970, he requested 
on May, 25, 1970 that Congress jppropriate $150, million under six existing OE and 
GEO legislative authorities^ tha*; coulfi be used immediately to h^p school district^ 
that were^'undergoing desegrega ion. Althpu^ the Congress is still considering the 
basic legislation, it did approiJi^ate one- half the^ supplemental money requested 
($75 million) m August 18, 197C 
Bfecaijse of the unusual way 



and thereby estabiisHed ESAP. 



tjie existing program was authorized and initiated, 
there 'are several special factors that affect^e iSSAP activities being evaluated by 



ads study: , 



A. 



Vhe 



(1) .The Education Profepsions Development Act, Part] 

(2) Ti^e Cooperative Re^s^arch Act./ ' 

(3) ThQ pivil Rights 'Acit of 1964; Title IV. ■ 

(4) The Element^ and Secondly Education Act of 196k Section 807. 

(5) The Elementary and Secondary Education JVdmendmfents of ^967, 
- Section 402. | / 

(6^ /The Economic Oppc^rtunity Actoof 1964, Title iL 



(1) Since no lofflslation has been enacted to establish objec- 
tives, requirements, etc., the applicable regulations 
are thoso written by DHEW and published in the Code of 
Federal llcgulations as Titie 45, Part 181. 

(2) Although Uio six statutory provisions being utilized have 
their own limitations, their combination and interpretation 
within ESAP regulations follow a comprehensive group of 
activities to be supported. • 

(3) Although the proposed legislation is intended to attack both 
d ^ juye and de facto segregation, ESAF only deals directly 

^ wuh de jure situation (i. e. , Category i above). The fcritical 
* ^ problem was thought to be the districts just beginning to 
operate under changed legal requirements. 

\ , 

The ESAP regulations were published in the Federal Register on Angus 
1970. and school districts iirged to apply for grants. Special, regional briefing 
were conducted and special processing procedures yfiere established in tt^ ^Hkw 
Regional Offices so that fimds could be channeled to the districts as fast as possible 
in order to be of holp in the early part of the 'school year. By Stepteniber .IB, l^VO, \ 
301 districts had been granted abdftt $17.2 niillion. Funds ' were Wocated td statejs.,^ ^ 
by formula and then liie »pitle IV Divison of the relevant DHEW Reponal Of£i<L , ^ . 
awarded individual project grants. By December 10, 1970 (the cutVoff date used ^ 
for sample seledtion in this study), 875 districts had been granted almost $60\Million. 




Jib expected to be granted direclfty to about 900 local sdiool ! 

districts^ during FY 71. The remainder, of jlthe $75 miliion appropriation will hfi ^ 

^ - I ' ill \ ■ . \ 

used for grants to cori munity groups (the sbj-cdll^d 10 ^ejrcent program) and for DHEW 

administration costs. Y ' ^ - 1 x " // i 



> administration costs. 

The general pifrpo'se of ESAP is evident from the follpvvdng words of the auth( 
' rizing regulations: , , ^ 

• ^ The puri)oso of the emergency assistance to m made available. . . 
is to meet special needs incident tq the elimination of racial segregation 
' and discrimination among students ^d faculty In (elementary and secon- 
j dary schools by contributing to the co'gts of new or expanded activities— 
] designed to ,achirve succosgful desegregation and the elimination of all 
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^ ' forms of discrimination In the schools on the basis of students or faiulty 
being members of a minority group. . ' \ 

As a result of such broad wording, an extremely wide variety of educational activ- 
ities are allowed to be funded under ESAP grants. The assumption is made that 
the local district knows best what it needs to help its desegregation process a^c^, 
therefore, that the district has the responsibility to decide what activities\should be 
undertaken. The Regional DREW Office was to verify, that decision when reviewing ' 
ESAP applications, but the pressures of quick tum-arouhd (the goal of 36 hours was 
usually met) meant that almost no reg^ested activities were judged unsuitable (except 
for air-conditioning requests, for example). The regulation^ and the applications 
present a long list of examples of activities and subactivities that were authorized. ^ 
The list is too long to present here, but the main groupings wer^ as follows: 




1. Part 181.2, Titie 45, of Code of Federal Regulations. 

2. RMC has adopted the following ESAP definitions for#consistency within 
this study^ It is believed they are compatible with the definitions used within X)E 
and most field personnel. * , > ^ jf • ♦ 

Program: used to describe the overall ESA prograAiTTri' total, 
the program consil^ts of grants- about ^00 school\ 
districts. ' *^ 

Project/: used .to describe the package of* ESAP fu|iictio]|is fund^^d 
at a speeifib school district. 

Activity L used to describe the ESAP functions of a pariibular 
type being carried out within a school district. P'or 
^ example, a teachpr training activity in Mve scl^ools 

of Richmond, Virginia. The ESAP project in a paSc- 
* ^cular \ school district may include om or more 
activities. ^ 

Siibactlvity: used to describe syfecategoribs withiif eaph broad 

activity category,/ For example, thi teacher train-, 
ing activity might have two iubactiviiticsf an in- ! 
service program to Improve human delations an(^ 
a demonstration program for lndividualiz<^d instini- 
ctions/Tcam Teaching. * ' ~ 
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(1) Special Community Programs>s 

(2) Special f>upil Persomiel Services"(ihcluding remedial services), 

(3) Special Curriculum Reyision Programs, . 

(4) Special Teacher^ Preparation Program, . > . ^ 

(5) Special Student- to- Student Programs, 

(6) Special Comprehensive Planning and Logistical Support, and • , ^ 

(7) Other ESAP Programs. ' ' . 

NEED FOR EVALUATION ^ . ^ * / 

' There are several reasons why ESAP being evaluated at this time. First is 
the .need for information and guidance for policy decisions about ESAP by OE and 
^ DREW staff. Examples of policy decisions that coiUd be influenced aije allocation 
\or rei^location of resources, drafting of guidelines and regulations, addition or > 
deletidn" of aiithorized ESAP activities, and specification of evaluation criteria and 



^ prdcbdu^es for Esil^ projects. Note that this need does not require ma^ng judg- 
,^ nients or decisions' about individual school districts; the emphasis is on policy 
I decisjti^ns that affect ESAP as a whole. 



^ , Th^^ stecond purpose is to provide inputs to the legislative process. Depending 
^ on tilningjt i^fotmation about effectiveness and efficiency of existing ESAP activities 
coul\i and should be a factor in shaping th^ final details of the pending $1. 5 billion 
legislation and aid pro-am. \ . , 

, A third reason for the study is to meetwthe requirements of the ESAP* regulations 
for an bvaluatibn by an independent organization—i.e. , one not connected with any 
' school district receiving funds oV to the parts of OE and DREW that administer or 
\m^tor^flie^E^P activities. ESAP, and school desegregation in general, is a very 
onti^oversial subject. It is impo^rtant to have an uijbfased, objei^ive research de- 
sign if the results of the evaluation are to be credible and useful. 

A brief word is needed about the relationship of this evaluation of ESAP to the 
many oilier evaluations that also exist. As stated, this eviluatioi|i is the independent, 
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outside evaluation required by the authorizing regulations and is sponsored by the 
central planning and evaluation agency within OE. The regional staff of the agency 
witiun OEthat administers ESAP (the revision of Equal Educational Opportunity, 
DEEq), kdit^ and monitorse^h ESAP grant, but these are the sapie people that 
helped deUgn many ESAP projects and apprpved all of them before funding could 
occur. The Office of cW Rights (OCR) of DHEW also monitors each ESAP 
school district (p well as Wers) to see that districts are fulfilling all relevant" 
civil righ^, regulations (incljding the special specifications of ESAP regulations) . 
Congress itself has also iJiitiated a. special audit of a sample of ESAP grants 
. yi&ts agency, the General Accounting Office (GAG). Although their initial focus- 
was the policies and procedurfes for approving ESAP grants, GAO is 'now apparently 
examining the ESAP activities^d'suba>>^vities.Qf specific school districts. , 

^ Addition, there have been pev^ral other evaluations and audits by private 
divil rights groups. Title IV desegregation ctentefs, congressional committee staff 
■ niembers, and State Education Department stiff. There is no question most ESAP 

dlistricts feel ov<Sr-evaluated (and often harassed), and this may be a . factor in the 
, r^Vcfion of some districts to the RMC survey". ' Based on present lm84ledge, none 
.of the othdr evaluations has tnter\tiewed ESAP participants (teachers, 'pr|nicipals, 
'/etc.), either systematically or in sufficient numbers. 

PURPOSE OF THIS STUDY • ^ • " , " 

' ■ - • ■ ' \' ■ . : 

• ^ was discussed above, the ESA program states that it is' directed toward 
specif \ieeds incident to desegregation. T(H|hip etild, it iSs providing assistance 
^designee! to moke the elimination of de jure radial s6^egation in the public schools - 
A successful and smooth process. The purposi of this indep^jdgg/assessment by 
- RMC is^ to evaluate the effectiveness of ESAJP^that is, to assess the effectiveness 
m meeting program goals. m 

L ' ' J \ \\ I 

For putposes of evaluation it is tmfor^ate that rripre specihc and measurali^c 
' objectives for the Emergency Schooji Assistance Program were not established t^d^ . 
published, but such was not the ca^e. ^ " ' ^' 

I . V . 

o , . . . . " ' ■ - ' < • • 
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Nl^dbperation with OE, 'the^f (Allowing several specific objectives wer^ defined 
to provide guidance during conduct of this assessment of ESAP. 

(I) Verify that ESAP projects are being conducted and determine whether 
chaiiges in objectives or activities have occurred from original 
plans. . ^ 

• (2) Assess the effectiveness of ESAP on the foll9Wing bases: 

• Has the Emergency School Assistance Program 

• acQomplished its goals? / 

• What social and educational effects have been 
\ produced by the several types of ESAP activities? , 

• Have ESAP projects had a meaningful role in the 
desegregation process (including specific dese- 
gregation plans, if relevant)? 

* (3) Assess the effectiveness of local project management in con- 

tributing to successful operatic^ of ESAP projects. 

(4) Assess the utility of the technical assistance provided by 
Federal ESAP staff to Ic^cal school districts; ^ 

This study is also intended to not examine several issues (except as they relate 
to the above objectives),^ in most cases because they are th6 subject of other studies 
or agencies. The procedutes and process used by both central and regional OE 
ste^to allocate ESAP funds and select ESAP projects were considered outside the 
scope of this study .^Furthermore, altiiough individual ESAP projects in a sample 
of districts were exaniined to obtain representative information about overall * 
ESAP activities, effectiv^]^ass, and problems, it was not intended to pronounce 
jud^ents about thfe actions of particular school districts. In addition, the 
checking for legal compUance by lu^ks with court orders ox HEW desegregation 

( regulations was avoided, partly because other agencies perfonn that function and 
partly^to increase the probability of getting local cooperation in obtaining the more 
iinportant information affecting ESAP effecfiveness.* KMC didjiot attempt to accu- 

1 rateiy obS^^ryfe or measure the' degree of racial intepation in schools or classrooms i 



/ 

\ , . 



1. In fact, the need to assure inony^ity to individual respondents precluded 
the fcedb/icJc of ESAP assessmcrits lo c^h LEA as originally desired. 



since that information had already been collected by the Office of Civil-Rights (OCR). 
Of courge, the precise auditing of accounting ^and fin^cial records is the function 
of internal government agencies. * 

The -RFP established two key goals '^r thrusts of this evaluati- 
strong iiifluence on the pxtrposes and ^the' resultant research design. 

•% ' s ^ ' \ 

(1) Independence. The evaluation should be conducted on^ontract 

by a non-govermnent organizatiofi"not sel^ted or responsible 

• to the ESAP pij3gram staff 0* the scl^ool districts. Moreover, 
the design and conduct of this evaluation should be independent 
of other eval^ons of ESAEr^though available information 

' from other ESAP studies should4)e considered when preparing, 
conclusions and Recommendations. 

(2) Timeliness. This evaluation m clearly short-term in nature,' 
primarily because ESAP pr^ects vill have been operating'fbr 
such short periods of time (somelhave yet J» start implem^-' ^ 
tation). Therefore, the focus wil^e on shor^^S^ or. Imme- 
diate effects resulting from ESAP prSj^is. A aricond impor- 
tant aspect^of timeliness is the short performance schedule. 

^ To provide majdmum use of results front the evaluation study 

* in the federal decision-making pyobess, learj/^tradi^ qf 
information (even tiiough prelm>mary) was ehcoura^d. 

' ^ s - . ^ * ' * ^ 

RESEARCH DESIGN • ' 

, The research design being used to assess ESAP follow^ very closely that 
specified by OE in the Request for Proposal (RtP). Althougb'tlie authorizing- 
regulations imply that all ESAP prbjects should be subjept to an outside evaluation, 
OE has deemed it reasonable an^i proper to examine a sample of the ESAP projects. 

A two-phased approach (in parallel) is being used to obtain information needed 
to fulfill the severajl study objectives discussed in the previous, section. Alt|iough 
the broad purposes of these ccJpctiijrent phases' are similar, it is important i 
clearly distinguish between thenl since the dat^ collection and an^sis methodologyi 
are considerably different. ^ 
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Phase I is a brbad survey of 252 school districts, whifeh < 

is about 30 percent pf those receiving ESAP funds. A s^tratified,, 
♦ 

■ ■ . ,/■- 
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.random samW was selected to obtain data rei)xiesentative 
of tiie total prc^ram. Field teams visiting each di^tr^ct will * 
admini$ter highljr&tnictured interviews to ESAP pro jebt direc- 
tors, pkncipala^^ea^ers, and students. , A field staff dSabout : \ 
^ 70 persons will\^ve c^pleted over 10,000 interviews at me 
close of \fie^ld worOo^JPhase L 1 In additioVA brief mail * 
. questionnaire will be sent to all members of -the E5AP Bi- racial 
Advisorj/Committee (BRAC) in each sample district, OE has 
recently asked th'at this BRAC survey be widened t6 cover the- 
comjlittees in all 900 districts. An integral part of Phase I is 
the cdding, processing and analyzing of all returned information 
so that answers can be formulated to the questions implied by 
the study objectives. 

Phase n is an intensive study of twenty selected ESAP dis- . 
tricts, while also seeking information that will assist in fulfilling 
the JItudy objectives already discussed. Phase n will attempt to 
identify and document ESAP activities that hav6 successfully 
aided the desegregation process, while several common data 
collection instruments will be used in the 20 districts, the basic 
thrust for Phase U is an in-depth, case study examination of the 
ESAP activities in an LEA, Jhus allowing concentration on the 
special chara^rigtScs'^^leaS^ situation. 

Details of the researcl^design for each Phase will be discussed in the appropri-. 
ate chapters' of this report 

One important aspect of V^psearch design that significantly affects all parts of 

* i 

the study is the need to rely heavily on the views aijd claims of affected personnel 
in the school system. These claimsj will be suppleinented by observations and 
conclusions of our 'field staff; but there is little opportunity for direct measurement 
of performance-based effebts tl^at could be influenced by ESAP activiK^. f^e 
timing of ESAP and tJWs survey means tha^^(l) the short period of ESAP impli- 
na^entation has allowed little opportunity for accumulative or long-term effects to 
be felt, and (2) the mid-semester time for mosi data collection precludes the 
availability of end-of- school-year measiirem ents and assessments. Moreover, 



J, Althou^ a second wave of interviews w^»aliSapl3inn"ed in some of these districts 
in May, this secopd wave was'dropped from the plans in light of the .delayed start of tlie 
, stadyjs^dJS^ siiort elapsed tim 



considerable past research wdth performance-: based measuremente such as student 
achievement and teaching performance indicate they probably have little usefulness 
in such evaluations. Theref6re> as suggested by the RFP, this^stuc^^ is mainly . 
utiMzmg a pei'solial interview approach to data collection, which obtains informa- 
tion about expectations, attitudes, opinions, and perceived results. 

ORGANIZATION OF INTERIM REPORT / 

V/' , ' .H . • -, . 

iThe following chapters present in some detail the development of the atujlj^ 
effort up to the present time as w^ll as the planned analysis • (Somewhat greater . 



emphasis is placed op Phase I than Phase n because Phase I started earlier.) 
Cnapjer 2 presents a discussion of the Phase I'evaluation and data collection c 



plans. 



Chapter 3 presents the analysis plan for the data from the'Ph^e.I s^arvey. The 
Phase II effort, current status, and evaluation plans are discussed in Chapter 4. 
Ajjipendix A contains layouts of the initial tabidations that will be produced as part 
of the Phase I analysis plan.. Appendixes presents, a copy of the current version of 
the questionnaire to be mailed to the Bl-racial Advisory ComS^tted. 



10 



o 

fir 



16 



EVALUATION AND. DATA COLLECTION 
DESIGN AND IMPI,E MENTATION 

introdugtTon " . . , , . 

• . ■ • ■ • •/ ■ ' 

This chapter, in five major sections,. covers the development of thfe evaluation 
design and the implementation of tiie data collection effort. It also provides ^ status 
report Snathe data collection phases. • • " I ' 

Tlie first section covers the rationale for the deslgi^ of the four major diata col- 
lection instruments: questionnaires for tiig ESAP project directqr, principals, 
teachers, and students. In addition, a shoi^r instrument is be|tog developed for 
members of the district's Bi-Racial Adfisory'Committee\(B^lAp), .The development 

^ ' r ' * • \ ^ I ' 

df measures by \\iUch ESAP achievements could be assessed played a major role, 

addition, the sensitive nature of the problem bei^ studied dictated that the 
questions be carefully framed ^d asked. ^ — 




The secQnd section is devoted to a discussion of the sanafile selecttonf '^^^Rfe^ 

selection procedures are tiigroughly dodmnented along wdth the^ interview verification 

plan and re^^ts, ' . \, . ' " • ^ ' • * • ' * • - • 

The traln^ of \he^j[j|^erviewerB is covered ii^the third section of this chapter, 

^ Major items included are the traihiiig session itself and the mech^sms by which 

the interview team results^^re^nonitored and corrected in tiie field, ' 

ilie fourth section discusi§es ihe organization of the field effort and reports on ' 
,1' • li * • • . ■ 

the;status of t;he field work as of April 1971, A discussion of problems encountered 

and'oiir expectations about the completion of this part of the data collection process 

are cdyere^. ' . , ^ , 

O |- ' ■ '''in - ^ - - ^ ■ 
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The statu^of the collecteJi .^ata is discussed in the fifth and.flnal section. The 
coding and editing processes arte icovered in depth\with some general impressions } 
of data quality being presented. 

' • -V ■ ^ ■ 

INSTRUl^T DESIGN ^ND EVALlWi^ION PLAN 

A 

General Considerations 

Three general considerations have had sign^^ant effec^(M>n the design of the 

; . \ t v ' ' 

data collection in?tri^ents. The first of these w|as the requirement to meet the 
study objectives by addressing and measuring the^^attitudes, behavior, and educa- 
tfonal performance of the participant^. It \yas necessary to develop and treat 
analytically specific behavioral measures and, at the same time, place the respon- ' ' 
dents* perceptions about behavioral changes in a meaningful context. This was ^ 
particularly difficiilt due to tie highly sensitive natute of the problems and questions 
dealing with any aspect of desegregation or its effects. We had to- address ESAP 
and black- white interac^ons squarely but without bringing* about an antagonistic, pur- 

, poaefuDy deceptivei or overly c^mproniising-sftuatibn.^ / ' , ' * 

The second major factor was the time elemWit involved. In many LkAsJdese- 
gregation and the initiation of ESAP.activities have occiirrctd almost siijaultaneously. 

^ This, of cotiTBe, 'Is an important reason *for m^dpg vfi^ clear'^jbur concesm with* 

assessing the impact of the ESAP activities rather thaa measmring the toctent of 

^ ^ / : • , V' •"^^ ■ 

desegregation itself. 'Since many of the LEAs receiving ESAP funds hav6 not had 

significant p*rior experience with desegreption and its problei^ the .analysis 
could not rely "on comparisions of the. magnitude of problems arisl^ 'frdm desegre- 
gation or educational pctformance before and after the introduction <3f ESAP activities. 
Another implicatfen of the time element is the lilnited length of time that, ESAP 
funds have been available. The foiir instruments^ are designed to measure changes in 
desegregaj^n- related problems, and behavior to the extent tiiey haveoccjurred 
thus far. While it is not possible in all cases to assess the magnitude of changes, 
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it is feasAle ta d&tect the participant s view ^bout the direction of change, and, 
if some cjia^ge i4 perceived* the cause of this change • ^ 
\ . The; t&ird area 6f concern was to malce it clear, to the respondents that our 
effort was not ^fiigned to check oa the 'compliance of tl^e particul'ar ;^EA with their 
desegregation^ plaij.^ Thus, we triad^to distinguish ourselves irom c^er groups'by? 
not asking qi^stions or checking^ activities jrelated to bivil rights legislation. In 
an analogous^ vei?i, we also tjAed to steer clear o^^asking miestions regamAj^ the 
■day-to-day eoAduct^of s.chool operations. Thus^ nc^ check on bus routes'jor seating 
assigT?mentk in classrooms or rj^gulations was made. By stressing this concern, 
we have.med to assure all respondents that we are simply trying to assess the 
impact cJf eSAP on e|asing the problem of desegregations- -not that we arejchecking 

up on them. - \ 

♦ . ' - - 

Desifen. Rationale • « ■ ' ''^ * 



The means for collecting fouivmajar' types of data were incorporated into the 

inpl^pmieht i^e^i^. The first was the establishment of tiespondent's problem- 
. ■ ■ i \ . / i , - . • ^ 

solving ; .tfituded;; ^'Problem'' niust h'ere be v6ry carefully defined. The concern 

is with those Sthool situation problems— both classroom and non-classroom— wMch 

have ariben as a result of desegregation and Which dan be addressed by the^ESAP ' 

, ac^vitife^*. * The. second area pf interest is the incidence of^pqrceived change in' 
behavior with regard to these problem areas. Third, any association between the 

• perceived behavioral changes and specific ESAP activities must be estj/blislied, 
'The atructui^ of these queiStions addressing change must clearlj distinguish 

' between chaage attributable to the^ ESAP program *and that change which has to/'^- 
ease the impJ^ct of desegregation on schools, perceived change that resulted from 
the desegregation process itself, or prior to the introduction of the ESAP activities 
in .the district, njoist be assessed to avoid overly crediting ESAP with bringing about 
change. ^/ ^ 

- The foux^th area deals with tl^e descriptive dataabout the specific ESAP activities 
. ii^ operation in tho^various schools and LEAsv . . 
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,WJthin thi3 set of problem ar^ats, the respondents' expectations with jrespecft to 
. dealing witli specific difficiitjes in these areas ^re to Jje established. That .is, 
pajrtipipants are to be classified according, to* whether they approached the solution 
of classroom problems of racial differences in achievement' levels, poor communi- 
cations between black students and white teachers and b^tw^ai white Students aiid , 
I black teachers, or heavy absenteeism" with an easy pr difficulty fo^e of refer^cp/ 
m effect, we are trying to determine Lf the participark beg^ the fall term^with the 
^ expectatioln that classroom prijblems could be easily solved or whether they would 
l^e difficult to solve. . The same approach was taHou^on the non-classroom problems 
arising out of qesegr^gation. Thus, information on the existe^og or magnitude of 
the problems stemmii^ from desegregation is not sought;. rather, we are seeking 
information on their expectationsT of dealing \vith or solving, these problems. After 
all, ESAP is directed at solving or easing of problems. ^ 

It must be noted that subsequent qhange is to be,related^to the exoectations of 
LEA personnel. Their prior experience with desegregation may, be ffimited or non- 
existpnt. For school districts with schools integrating for" the first time', the appli- 
cations for ESAP funds may well be addressed toward solving the problems tiiat 




Met that pange, or lack of chan^ as perceived by a participant'-since-the initiation 
o3f ESAP activities, may b^ affecfeS fey his approach. - \Thus, ^e respondent's, 
approach to problem solving becomes an important explanatory variable for per- 
ceived behavioral change. If certain activitae^ turn out to be th^$ most productive ^ 
to those with a difficult- to- solve outlook, this information may be of considerable 
(interest to OE program personnel. - ' .' 

For those L6As that are using the ESAP fimds t6 perpetutate a previously 
started activity or to 'continue a previously desegregated environment, the same 
expectations are sought,' although in those caseq the problems may well be real' 
and. observable. 
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^ Perceived Chknges in Behavior > .We seek next to determlije if changes in 
.Ippecific biehavior or performance have been perceived by th^ respondejgi. The 
• chamges we are interested in are th^sa.which are measured from the nearly part ^ 

qf the schooLyear, although specific ESAP activities are not yet directly mentioned. 

V ' • *• \ " ' j\ '] ' * : 

, Tha specific changes in behavitor jvpre d^Yelopea from the list of prpblem areas. 

M ' \ ^ Ml \ /I ^ " ■ * 

^ It|Voi}ld be desirable if each of the bena^l^ chajoges could be measured precisely. 

For e5Sahiple,.$cademifc achievement pan be^ measured tl^rough testing; class. 

p^ici^ation ca^ be observed and quantified by noting relative numbers o^Vespjo: 

by race. goweV^r, in thi^ phase of thd study, t^ere is neithegr timq.nor base. data 

. ■' ' \ \ • ' \ .V 

to. fully utilize these 'types of measurjements. The intQrview.ers are pot in a school 

or district long enough to produce' reliable numbers or incident cw^unts. Limited 

time .and r^souxces preclude examin^ition of school recorc^p, although interviewers 

will be fijiliy familiar with project, applications. For the above Reasons, the current 

effort focused only Upon estalDlishing if change has b^en perceived and th^e direction 

of this change, f ^\ ^ . f . . > ^ « 

. - . / - . ' / . ^ 1. • j - . 

In the classroom probl§pa area, for example, we seek to find out if the per^ 
celved gap in achievement levels betweien the black and .wliite students is lessening 
ox widening or showing n9|chan^e since the eajvly part of the school yeai:* W^e seek 
additionally to find out if the respondents fe.el that black Students are respoiraing 
more freely, less freely, ox, with no change to white teacher$ since the J^eginning 
of the schooil year,' and so forth. Thus, we seek the cflJr6ction of chaii^? In spec^ic 
behavior or performance. • * 

Not only are the specific behavioral chaiiges related to the problem areas' as 

di^cussec^above, fjjiey can. also be expected to be iiifhiericed by one or more of the ' 

ESAP activities if these at^tivities are effective. Thus, after the participant's 

' perceptions of behavioral charge are examined, association of theSe changes with 

* *■ ' ' * ' * 

Specific ESAP activities can bo Investigtalid. ' ' . ' ' 



o ■ 

JC . • .15 



Association Mth ESAP Abfivities,^ l^e interview^t must elicit from the 
respbhdent what he perceives 'to be the caiise or causes .of each percei\Eed behavioral 
ch^ge* Thus, if a participant notes that black student^ are indeed responding more 
freejy to white, teachers ii^th^ classroom, '"the'interviewer probes for the reasons. 
He would expi^t that activitie s being funded by^ESAP will be associated with some of 
I these chaiiges—pbssibiy movi\ often by dire^ctors and'prihcipals than by teachers \ 
and students. The probing by the interviewer does not try to elicit non ESAf>- related 
response— it obly allows for tlie response to arise if the respondent considers iiio\ 
be of importaacV. "Thus, chaiige is associated with an ES^P activity through thi^ 
indirect approach. ^ This section of the /interview is open-ended to give ad much 
latitude as possible to allow tlie participant to identify those activities or events 
in his school that he/feels have been of some influence on changing pehavior. * 

We now have the basis for'Sigsociatiag per.ceived change with specific ESAP 
activities at sfcho(^^/ ;if the BSAP activity is .related to the chang^ in a manner that 
called its attentioi^ lo the participant,* it will become evident. DijEferences in reasons 
for change among directors, 'brincipals,.*t€tachep and students can be identified. 
If factor? other than ESAP ^r6 considered to be associated with ch^^e or if ^ 
ESAP activity in conjunction with.another non-^SAP occurrence appears delated 



to the change, it can be ho 



Descriptive Data. The fourth and. final major area of interest incppporated 
^in, the instrjnnenf design is desscriptive data about specific ESAP activities. The 
activities tak^ plac? a^ the jDarticular school will be identified by the proj^. 
directoj;^ 'Afes'e dak are necessary to stlppprt the verification and project-maiiage*t 
ment'obj^i^V^s.of the study. In additidh, the lengtit of operation of the activity will 
be obtained. The type of queistioning varies because directors, principals, knd 
te&chers can answer differcnn parts of the verification, \ management, and descrip;-. 
tive needs. Much of the ba$ic vex^cation data is solicited from the director: - • 
how many teachers' aides ha\e been hired, what materials have b|^n purchased,. 
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and so forth. TTie principals are also a^kel 



It 



the personnel and materials 
'e being used' in his school. 



m|ide available tp the district by the ESA^ 

F<^r eac^ bij^oa^ ac^ty Category, a set of atjitudinal question^ has .been in- 



cluded to tr| t;b assess the junpact of the pte^nce of the ESAP activity in the 
school on the ii^espondei^t's attitude. These attitude questions are aslced of all 
respondents regardles 3 of their Knowledge pf the particular activity. 
Y The cjegre'e of p^jjicipation iii the formulation of fundipg, Applications by the 



spondeiAs^.ana thfe lir es of communication irom teacher tk) principal to director, 



' i^cogniiingl^e possibility of problems ik bl^-white intei^jjtions, are a][so 

investigated' afe a means of determining h^w r^ponsive and, thus how effe9kve, 
i ' project man^gem^|i't apppars to be> 

This evaluati()n approach is desigiiW ^to wclrk froi^ tlie problems of desegre- 



gation to the ?pe<^ific dccomplisl^ents,^. ^Ipercielved by the^participailts, of ESAP 
ac|ivities. Considerable effort h^s beeA made tb avoid becomin^boggid down in 
at^tudes towar^ Irapial issues an^ attit^dW toward desegregation ^tself. The dir- \ 
ecjtion, throughpjlit^ is toward the /specific Vr(Jbl|bks b^'ing. dealt wi^ by ESAP « , 
/ctivities, the Changes associated with thel^ specific problem area's, and the ' 
attribution of these changes to specie E^AP activities. Respondents are cate- 
gpvized on the|)asis of solving problems and .their approach to these problems— 
not to' race i^elations, per se. ;rhe'time frame is clearly confined to the same time 
as tb^ ESAP activities themselves. - ' * ^ ; . ^ . » 

By collecting EISAP impact data in this manner, the Office of Education will 
receive results t)hat Will allow them to ^ss^Ss the relative effect on desegregation - 
of each of the seW^ ESAP activities. . By IsolicitLpg infbrmati9n on other causal 
factors xelatilig t^ thl improvement \of the integrated situation, the effe6t can be 
viewed ih proi^er perspective/. If ceitain non^ESAP activities are seen as parti- 
cularly ^elpful^, they^^^d^be coni^idered fbjc, inclusion as future ESAP activites.^^ 
Further, since some actMties haA)ei been operating for different lengths of time, 
those activities with the q^ckeist impact can be isolated. .Again, it is important' 
t^at perceived changes'be related to ESAP only. • 
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Data Collection Procedures 



1 of Instritjnen 



^ Th^re ^e four sepajrate survey ifl^trument? for thf different persWiel\cate- 
gories in the l|EAs: ESAP project directors, school principals, teachers, an^ 
students. Fbr/ail^iit the student category, the ba^ic desigp of these instrumants 
at. In the 



is v6ry sin^ila; 




case of the project director,' additional linkterial is re 



^ informajtMAlabout the administration and impleme|i|ayon of -the 
activitife^AxBecause of hik more limited' base pf knowledge, tlje 
student questionjtiaire Is much simpler ahd does not include separate sectioni 
for. speciffc ESAP activities. M * . ^ 

In addition, k mail questionnaire is .cirrently being prepareji to elicit, responses 
from members of the ESAP Bi-Racial A^yislpty .Cpmmittee (BRAC) in aU ^ie affected 



LEAq. ^^This fo 
.Whole arid the 
/this ins 




YiM obtain data on the d^^ 
Wdual member in partiqularj- 



^eelof i 



ent is^hovl|n in Appendix! B. 
tostrid^nt Laj^oiit and Question Framing 

The director, principal, and teachers questionnaires are enough s^ike to be * 




involveijient of the conunittee as a 
copy of thamo it recent c^aft of 



discussed together. ^'The'^instruments are composed of three.main p^rls. ^The first 
section combines the questions dealing with the problem- sol vmg expectations, thai 
l/erceived changes, and reasons for the changes. This information is olM;ained from ^ 
' L series of about two dozen questions. Each .''expectation" question is designed tb • 
'"depict a proSleni that can be addressed by ajhESAP activity. Each ''change'' ques- 
tion has a direct counterpart With an "exi)ectation" question. In addition, the questions 
can be cate^rized broadly into the cl^ssroom/non-classroom breakdown discussed 
earlier as well as into persoifcel relationships and academic perfomance. . 

2^ this section of the instrument the key dependent variables of this analys 
are found. They are the incidence of change (positive or negative) and the assci^J^ 
tlon of an ESAP activity with this change. The set of expectations are aggreg/i e^d ^ 
to produce an independent variable that will allow us to categorize the respondents 
^ Into their feelings with regard to being al)le to overcome the problems associated 
^Ith desegi;egation. t y 
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An indirect aRproach was selected for th^ expectation queslkons; |hat iSj, re 
pohdents werp asked how they bought teach^ifjsyfelt about certaia things .as 6pp(^|^d 
to how they felt. In Sensitive a[reas such as Megration, indirect techn< ues ha 
t^li^n shown to produce n^ore realistic answei^-s V^n the direct mode. The -^exp^tlt 
ation' (^ue§tiUs_were closed-ended w^th tou^ possibly answers between "very eaiv" 



^"1 



and 



}^^|j7jdi|ficultl regarding the soltition o| p'lj 
'ilJiVe^ard to ibese expectation^, the iiteri' 



iblems . 



iVegard 

respnndent, at the beginning of this schoc 



^iiteritiwas to determine wlhether the 
t'ear, anticipated that the problems 



\ 



Ukeiytp arise Wouja fee very difficult to'dfezjl Vith, difficult, ^asy, or very easy 
■ to ^eal with.j Jhis t^e of \^espons| 'is ne^^asy to e.Hci/as Wrppon^ent's 
reijent! experiec* may|infjlu^ce. hi^ xecall i'f past emectations. Nevertheless, ^ 
y lacking the oppp|tunitjS to'questioniryspp^^ on a b^fore-and-after basis, this / 
• ' v)^s tl^el only possible .approach. . Qufestipns Were phrased t*' remove any suggestion , 
1 tiat nljght make'the respondent feel thai he|wks,bei|a^ judged on the basis of what ; 
helsaij^e expecte^ as opposed ttrw^t actually ocqUrred. In ad(^ition, since "the . 
beginning of the* school year" marled, k very clear tiipie point for school personnel 



— .... ..^^^^^j. iiiaijw B. vcijf kij^ue pumt. lyr syiiuojL personnel 

as a relatively ea^sj^^point at which to focus for the recall of expec- 



and students 

tations wHereas a date within the ichool ^^ar, even if relatei to the implementation 
ef the ESAP activities in the distl/ict,^vould have be&Q more jaifficlilt with which to 
associate past feeliiigs. > I '\ 1 

■ ^ The "change" questibiis werj partly open-ended and were worded in a more 
4it'ect manner. Since we were asking a^b\i^ perceived change, we yakted the | 
responses to reflect what the respondent!^ had observed during, th^ ctlr^ent school i 

d to ao with specific pccu: 



A 



The changes all had to do with specific 
.in pjpinciple, n^easurablc, If a positive or 



occurrences or conditi^ili ,\ that were^ 
i^gative/ change wai^ volVieered by the 



fesijohdent, the interviewer probed to find JpW^ ihd* change Jas att^tlpted. In 



thislopantendod'pwtion, the relating of cha 
wag ac 

chjuigipwas ftssopiafcd with one|oy'|mor 



\^ |> in ESAP actmtyj |fWw}as related, " 
roijc guiclcd toward ^ayind ihat the • 




3 ESAPlactivitics. 
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I To cove^the instancei^wlLre a respondent offered Several "no changg" answers 



for the set of '^^hinjge!' quemons, another question, \^hich was aimed at ideterminlng 



the current status o^ co] 



I, 



was asked. Thus, we 



iitibhs of integration Ui the school 
would be able i o categotjlze perceived status at that schlool to j^ome ^xtent» 

The secorci major |Lf||of the lnstrunipnfcy|s aimed at finding o^t which EfSAP 
activities are being condu^^i in the district and which schools are[ affecie<;J, Once 
these data were obtained, the appro^iriate Vctivity-specific modulei coula be aske^,^ 
Orie question in this bei^iiori (Question 10 of the director's. instrument) is the key 
|p this proceed. The intlrvilwer must plicft from the project direotor what he calls^ 
the ESAP activities in hisldistrict. Since there are many kinds of tiacher trainii^ 



programs, \for example, a common term for whatUs going pn in the district is 
needed. It is necessary for J»vo reas^ 

First| me appropriate activity- specific mo(Mcs must be asked ©f the director 
and the othei respondei^ts in the LEA. Thus, the module for teachers' k^les is 
only relevant^ Whenjeach^rs^ aitjes are planned former are being used inithje LEA. 
Second, this informatibn naust be^ transferred to the principal and^t^cher instru- 
ments so the proper module(s) can l?e asked of thenl also. . / ' \ 

>rhus, the interviewer must be sure that he has-obt^'ined from the director a 
title which will be most likely to be known in the schools that have the activity ^ ' 
and that will allow for the proper set of questions to be asked. * 

The other parts of this question and the nSoduKs'are then used to provide the 
descriptive prop9rties of the activities. Thus, several independent variables are 
generated in this portion of the design. ,They include the activities in a school and 
LEA, the amount of funds being sp^nt on each a^ctiyifey, the length of time it hats b^en 
in operation! the number and type of persons employed in the implementation^of^e 
activity, any\ materials that have been purchased, and how the personnel and 
matei iajls ari being used. In additidn, a set of ittitude questions is asked to deter- 
mine Ihb respbndent's impression of how the activity has affected specific schcipl 
problems. I \ / 
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^IS'Ipewayj^s needed to'allow the pr(j)ject director to descrtt)e hik" activities 
and- j^ubac^ivMY as he best saw itl most of the questions, in this sectiin are open- 
end(!d. They afb mainly descriptive, however, -so that they could be a^ked in a 
straightfoiixikrd manner;. , ■ ' . - 

The' final part of t|ie qufeistionnaire design "serves to provide demographic data 
on the responlJent and coybrs the areas of technical as-sistanice and py4ject manage- 
ment. T^e ddmogrLhip^ta- are heeded to provide the independent Ivapiables such 
' as race) kge, ^^eri|nce in education, and. length of time spent in the^rticular LEA. 



of tech^cal assistance and project management, the questions are 




Most o: 



^ • fijithe ^reftp 

Idesigne^ to elliit the Wbressions Of the effeptiveness of technical assistance arid the 
extent tJ wliiih the respondent has played a role in the planning and implementation * 
of thp. ESA']^ ] ^rcJject in thfeir LEA or school. 

hfe C[uestiond ijQthis last sectipn are closedrended because the response- 
categofies^^cJuld be predefined. Only the nature of the technical assistance and a 
question regirding ESAP implementation difficulties were opeti- ended. 

Waddition to the data collected diiiring t^ interviews, some data made available 
by th^ Offic^ of CivH Rights are being^ fentered into the data base. Jhose data include 
ESAP LEA and school identification numbers, the number of minority and non-minority 
Mudents, the percentof stud^ts reassigned, the urban/rural designation of the LEA. 
P and the breakdovm of the LEA's ESAP grant, as approved. 

The quebtionn^res were precede^ to speed up the keypunching and editing 
processes. Thus, for all the closed-ended questions, the code column and digit to 
be punclied were used as the response (that was dircied by the interviewer. Thus, 
much of the coining that tellsUhe keypuiicher what Ip punch has been done in the field . 
by the interviSwer. ' * ^ ' ' 

For the open-ended qu^Uons, categories have been created by RMC analysts 
" based upon ex^minilig i^everal hundred Icompleted questionnaires. There is one 
specfal considctration of not0 rcgardingi the open-ended coding. Due to the inter- 
relationship of |hj) EiSAJ^^tivitics and the "change" questions on any given ques- 
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tioimaire,in additl^to the director, principal,, teacher questionnaires as a set^ 
a common set opcodes has bleen established. Thus, if teachers' aides are as,signcd 
the code 5, then eyery pl^ce m the "change" se^of questions anfl every place^e activity 
modules that tethers' aides are mentioned, is coded a;s^ an 8, 

This is us4ful for several reasons. It allow|5rfor^sy tracking of an activity 
pugh a qi^sljionnaire and through an LEA. It bUows for easy matching of 
responses /to the ESAP acti^ties that the LEA is using. Thus, if positive change ^ 
is aWibi^ed toxeacher^' aides and'if that LEA puHhased teacher's aides, tl^ change 
can associated with the ESAP project/ Finally, ^the mi^ual coding process 1^ 
simp^fied because the coders know to code teachers' aides as an 8 every time they are 
encountered. Some 50 questions are to be cod^d usiaig this common code; 

The studeAt questionnaire^was designed ia^ different format mainly due to 
the fapt that much bf the "change" section a/d toowledge of Wap activities was 
not applicable to them. A short section on their expectatioks and the changes with > 
^regard to classroom Performance and behayibr was asked, however. \ ^ ^ 

By and large the ffladents were asked sdveral questions about attending classes 
in a dese^gpregated environment. The concepl was ^ try to pick up an impression 
of their attitudes about attending an integrated school. Therd were some specific 
multicultural and interracial questions asked, that are relatable to change and, in 
some instances, to ESAP activities. ' 

There is a short demographic section tha^ provides raqe, selc, age, ^. , 
and length of lime that they have lived in the UllEA area. These will be used as 
independent variable descriptors. • 



\ 



* ^ 

Pretest of Instruments and Ote Approv^drr ^^ 

' ' \ ' / ' 

The set of four instruments were pretested in two LEAs in Georgia. The pre- 

test| provided a basis upon which to test the series of questions to see if they were 

providing the necessiry information and to obtain estimates of interview tin^e ^d 

sample selection me ins. Bv and large the ptetest showed that the "expectaliolis" 



By and large the ptetest showed that the "expecta 



scl of questions at the 3e{ innirfg of the insirijmdnt^ were picking up diff crcn 
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person's approaches to solving the problems of desegregatiojn, T^e of behavioral 
change questions were sinula^rly found to pick up change7-both positive and negative- 
as -well as indication&^of no change, ^hus, the first part of tfie instrumeiit appeared 
to do what was desired. The length c^these qu6s((ions was r^du^ed. because the pre- 
test showed-ihem td be too long and'toWepetitii/e In nature. ' ' 

The^retest showed that the teacherr-toieY little'aboi^t the ESAP 'i^ctivities, which 
led to the l|iodification ofjthe activity- speciho^ for thejteacher instilment. 
The attitude questions in'the activity-specifi/mo^ul^^ wa^4ced well in the priest. 

Teachers and students of both races wew/interviewec^. ijhe school selection 
, procedures >w^rb not tested as only one school in each LEA was visited during the 
pretest by prapr arratigement* 

' The pretest \lso showed that some of the questions could be asked in a more 
direct mannerrFparticularly the student questions, so changes were made to reflect 
this finding* \ i , , " . 

The time required to admin^ter the questionnaires was found to be about one- 
^half hour for the teaser and student instruments, tliree-qtiarters hour for the , 
principals, and on^ an^<ine-quarter'hour for-the project dftrector, These^times 
dojnot include sample selection or traveling times. It was bn this basis that tfie 
plpmipg for the field operations and the number of teams was based. 

In late January 1971 a serie^ of meetings between RMCJ,\ OE, and'pMB per- 
sonnel began to prepare for the necessary 0MB approval of tiie survey ^instruments. 
0MB was kept apprised. of the development of the instrument^, frpm thl conceptual 
design through th^ final version. 

A supporting staten^ent was drafted to accompany^ the instiJum^nts* This state- 
ment outlined the^study rationale, instrument design, sample selection, analysis ^ 
plans, and the results of the pretest. 0MB approval was received on March 3, 1S7I, 
with the trained te^m leaders heading for the, field the next day. ' 



1 



'SA^IPLE SELECTION 

Two hundred fifty-two LEAs out of approximately 875 that were approved for 
ESAP funding by early December 1970 were to be surveyed in this outside evaluation. 
Within each LEA, an average of 3-1/2 schools were sampled with the principal, five 
teachers, -and five, students being interviewed at each school. Thus, there .were four 
stages of sample selepabn. First," LEAs were selected. Within each'selected LEA a 
^ample of schools wate chosen. Teachers were selected from the scjiools and, finally, 
students were selected from the classrooms of the teachers. The method and rationale 
•of sel6ctii)n are described for each stage. 



* 



Sample of LEAs . • ' 

^ The method of LEA sample selection was designe^> to ensure representation 
^ with respect to size of graat, ^eo'graphical regjon, percent minority, and percent 
^ of- students reassigned. The sample was also checked fi^ size with respect tc 
individual types of ESAP activities. In making -inference^ about all 875 LEX^from 
a simple randoi^ sample of size 252, |he 95 percent confidence intervalibout a 
measured proportion of 50 percent would b6 approximately 1 5 percen^ge points. 
^ J^presentation with respect to size of grant was obtained through Uratificatipn 
of LEA^ by.the 7 grant categories in use by the Office of Education. Stratification 1 
by grant size permitted sampUng of LEAsUn the larger grant categories at differential 
rates. Sampling in these categ6rie,s at higgler than average rates is desirable from 
the point ^Of. view of statistical .efficiency for estimates relating t6 pupils and funds. 
Since the! LEAs' with the larger grants kre more likely to have-a greater variety of 
activities, this Talso Enhances the sample distribution across aU types of projects. 
Thus, all LE4s in the top three grant categories have been included in the sample. 
The sampling fractions used for the other four grant categories, and for further 
^ampl^ng with the grant size strata, were similarly derived with design efiiclency ' 
and si nplicity of sampling error calculations in m|nd. The sampling fractions 
derived for each grant category imdeiwthis criterion are sliown- in Table 2-1 along, 
with the poj[^ulations and^nj^le sizes. ' 

T().obtai\i rcprc^ntation witji fespect to geographical region and percent minority, 
categories of LEA/were^crmcc(with rcspecjt t6vcach of llicse varies; and LEAs 
were cluster^ yithin gran}., slratS^cording to groups ofj llv.use categories. Percent 
minority was diVidcd into ton categories of ten percentage j)olifts each. States were ' 



divided Into five categories as Shown in Tible 2-2. ' ' oa 

ERIC \ . .^ ' ' *• ' 



Table 2-1 

/ tiEA SAMPLE DISTRIBUTION BY GRANT SIZE 





Grant Size 


Pp^jilation 


ocLxnpiing 
Fraction . 


Sample 
, Size 


$1,000,000 
^500,000-$999,999 
$200,Q04)-$499/999 . 
$100j()00-$199,999 
$ 50,000-$ 99,999. 
$ 25,000-$ 49,999 

0-$ 24,999 

tota 


11 

. 24 - , 
73 V 
173 
.238, 

l1 • '847 


1 . , 

y' 1 

. 1/2 


5 

11 
24 

36 , 

,\ 55 ;■ 

58 

60 - 



Table 2-2 



CATEcb^ZATION OF LEA^S 



/I r ■[ 


' „ ■ 


HI ■ * 


-v ' 

IV 


States . 


No.of\ 
LEAs\ 


I 
• 

'States ' 


NoiOf 
LEA^ 


. States . 


No.of- 
LEAs 


; ~States 


'.•.N6.'of 
LEAs 


Texas 

Oklahpma 

Arkansas 

Total ^' 

• 


101 \ 
10 
81 

192 


Kentucky-^ 
Maryland 
North Carol 
Vir^nia 

Total 


■ 

4 
/4 
ina 82. 

14'^ ■ 


Florida 
Tennessee 
South Carlolina 

Total 


57 
37 
68 

'162 


Georgia 
Loui§iana 

Tot^il 


148 
38 
156 


1 

1 ^ 

t 




* > 

31 . . : 


states ' 


< 

No. of ' • 

LEAs 


!Mississj{pf/i/ 
Alabama ; • ^ 

Total , 

• - / 


94 

, 65 

159 

• « * 



^ Table '2-4 

' LEA SAMPLE DISTRIBUTION B>f PERCENT MINORITY 



Percent 
. Minority \ 


•Grants less ' 
, than $200, OOt) 


y Grants^ $200,000 
\ and Above 

A; ^ 


Total 


' ' " r ; 




\ • 


6 


10-19 ' V 


22 


22 


20--29 ■ \ 


31 . 




40 


, 30-39 


48 


4 , 


52 


\ 40-49 ■■" 
50-59 


. 39 . 




50 ■• 


36' 


- 5 


41 


\ 66-69 - 
V 70-79 


1^ ■ . 




19 ■ 




5 


16 


' 80-89 


5 


1 ■ 


6. ' 


90-100 









- ,v ' - ■ , 

' . Table 2-5 



SAMPLE DISTRIBUTION BY JPERCENT REASSIGNED 



' . Percent 
Reassigned 


Grants less 
than $200, 000 ^ 


Grants $200, OPO 
and Aboy^ 


>Totil 

\ , 


^ 0-9 


■ . i-6' •■ ' 


8 


22 \ 


10-19 


17 


■ X 1 ■ ■ 


24" ^ 


20-29 ■ 


30 1 




8 


38 


30-39 


36 


■ \ 




44 


40-49 


, 30. ' • 






33 


. 50-59-^ . - 


27 ' 




5 


32 


60-69 - 


.17 


,0 


17 


. ' 70-79 


^4 > ' 


1 


■3 ' 


80-89 , 


' 4 


. • 0 \ ' 


■ 4 . ' 


90-100 


> 36 
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For design efficiency, two guidelines ^ere followed in forming clusters of 



LEAs: 



the total number of LEAs in a grant stratum were distributed as 
^ uniformly over ihe clusters^ possible. Specifically, the cluster , ; 
; was to contaih the number of LEAs required to produce a sample of / 
• or 6^ LEAs when tiie grant stratum 'sampling fraction was applie^. 

^ ter was formed so th^t it Avas as homogeneous^ as possible 
^th respec^to p^cent minority and geographical distributions. 

i ^ - 7 ^ 

Good representatidn with respect to percent of students reassigned was chained 
by ordering t he LEA s wiihin each cluster by percent reassigned and drawing a ^ 
Bystematic^^ample us^ a. random start* ' 

Following the&e sampling procedures, a random sample of 252 LEAs was 
drawn.. The distributions of the resulting sample of LEAs by state, hy percent 
minority, and by percent reassigned are shown' in Tables 2-3^ ^r4, and 2-5, 
respectively. * ' ' / 



\ 



Table 2-3 ' 



\ 



LEA SAMPLE DISTRIBUTION BY STATE 



l^tate 



labama 
Arkansas ^ 
Florida 
Georgia 
Kentucky 
Louisiana 
Maryland * 
Mississippi 
North Czlrolina 
Oklahoma 
South Carolina 
Tennessee 
Texas 
Virginia 



Grants less 
thaiv $20 0,000 



18 

3^ 

1 
12 

2 
22 
25 

1 
15 

9 
23 
18 



Grants $200,000 
•and Above 



2' 
0 

8 
5 

•• 0, 
5 
0 
1 
6 
0 
4 
3 

• 2 
3 



\ 



Total 



20 
18 
21 
41 

1 
17 

2 
23 
31 

1 
19 
12 

^3 



Sample of Schools ^ 

As specified by the Office of Education, a total of approximately 875 schoolsi 
from the 252 LEAs will be sampled* To ensure representat/oi with respect to 



grant ^iz^, the sample was stratified by this variable. Sampling rates for the 
stratayivere chosen in such a way as to enhance the statistical efficiency of esti- 
mated relating to pupils and funds. They are shown by str^ in Table 2-6, along 
with the estimated number of schools in the sample LEAs and the size of the sample 
of scljools. * • , * 



Table 2-6 

STRATIFICATION OF SAMPLE OF SCHOOLS 



/ 



, Grant Size 


\ Estimated 

Total ' ' ■ 
Schools 


Ayet^e 
School 
Sampling . 
Fraction 


-Sample 
■ of 
Schools \. 


/ ^1,600,000 ^. , 
500,0.00 - 999; 999 
200,000 - 499,999 
100,000 - 199; 999 
50,000 - \ 99; 999 
25,000 - 49,999, 
0 •< 1^4,999 


812 
608 
790 
534 
610 
361 
197 


1/8' 
,1/10> 
■ 1/10 
. 1/3 . 
1/3 . 
1/3. 
1/2 


' / 96 

''■ 65 ■ \ - 
93 

140 \ 
■205 \\ 
167 \ 
113 \ 


■ipTAL 


3,912 


1 - V 


879 



f The sampling rates that are shown in Table 2*^6 are^average sampling rates for 
the strata because not all LEAs within a stratum were sampled at the same rate. 
Bather, all LEAs of the same size were sampled at the same rate regardless of the 
grant size stratum they arc in. Table 2-7 shows a* portion of l^e cfistribution of 
LEAs by grant size and LEA'size. Also, it ^hows the sampling fraction by LEA size. 
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28 



Xhese are th6 actual sampjling fractio:tXS dsg^ in selecting the sample- For all i 
LEAs <)f three affected/schopl;^ or less, the. sampliBg rate was set go that one' 
school woi\ld be selected from each of these LEAs. The remaining sampling frac- 
tions .were set 'so that the overall sampling fractions, for the grant categories 
wpuld approximate those shown in Table 2r6. A maxinium sample of 16 schools 
from toy LEA was specified. To approximate the sainpling fraction of one-eighth 
for the $1,0)0,000 op more g^rant category, 16 schools had to be selected from all 
six U^ks m this category. \^ . ^ ' . 

The sdipple of schools in an LEA was selected 'from the total list of schools 
participating inthe'ESA prograiri, ^ven thoughi as /is e^qplained later, students j 
were not to be iiiterviewed in somte types of schools. The tptal ligt of schools in 
each LEA was divSfied into two categories: those that have a he&vy concentration 
of ESAP adrivftie? anid those that do not. The sample of schoolsVas selected prior 
to the interviewer's af-^ival at t^e LEA. Information on the current list of schools 
inthe ES^P project and the schools heavily involved with ESAp activities was veri- 
fled with the project directors by phone. \ 
\ Schortfs with a heavy concentration of ESAP activities were expecte^ ^o be in 
the minority. To ensure adequate representation of these schools with the largest 
amounts of ESAP funds, ^hey were sampled at a higher rate than thA other schools 
•involved' in the ESA program. This was accomplished by simply dr^Ihg half of 
the sample of schools from l^fae. list of schbols with a heavy concentration of programs 
and half from the remainder of schools recei^ ^SAP funds. .Thslntervfewers 
were instructed as to 'sj^ep^flcally which schoofl they wpuld visit in eadh district. 

Sample of Teachers , 

; In accordance with the objective of sampling to appVpximate optimum allocation, 
exactly five teacljers Were selected from each school in the sample. To ensure good 

representation of teachers directly involved in ESA? activites, two teachers werg 

■ / ' ' * - ■ . . ■ ■ 

■ . 7 • ■ . ■ ■ • . ; \ 



randomly selected 3from those directly iavolv^ in the progj^m^d three we;re ran- 




teachers~i»hat school. The 



tly iavolvjed i 

domly selected from the remainder of the li^lof 

^ selectionjvas done by the interviewers tie field kom lists obtained from th^ 
principal of the school. The procedure for Selection utilized the random- star^t 
fixed-interval technique and was provided jto th.e interviewer when he went into the 



field. " / ' '\ 

t : , ' ■ / -I U 

Sample Qf Students 



J , As with teachers, .five students werjj/selected from each school in the sample, 
with the exception of those schools whose^grade\span is less than the jtiftli grade ^ 
or those special schools (such as /correctional scnools or certain schools for the? 



to i ^ _ _ ^ 

j/si 



handicapped) where intervlew|ag /students was unfeasible. . One stujdent was randomly 
selected from the class roll ofje^ch classroom teacher in the sample. If the teacher 
has more than one class durlng'jthe day, the student was selected from the first 
class taught. 

If there were less than five , class room teachers in the saniijle, a classroom 
teacher in the sample was randomly selected ;^d a second student was then 
randomly selected from the clask of that teacher. To avoid excessive clustering 
of the sample of students,, no.mqre than two students were selected from me class 

1 ' \ ^ ' ' ' ■ n \ ' 

of any one teacher. If at least tvJ^o students had been chosen from the cls^ses ofi^ 
all classroom teachers in ^he sample and thy^e were still students to bejpitl^ed, the 
reniaining students were selected by first randomly picking teachers not in the 
sample and then randomly selecting students from the classes of these teachers^ 
The additional teachers were not interviewed. 

Verification of Interviews 



Project director, principal, and teacher interviews are being verified by ^ 
randomly selecting Samples of these individuals and checking, l)y telephone, their 
rcsponsei^ to a portion dijixe survey questions. Student interviews arc not being 
verified because of the difficulty in reaching the sliidonts. So that the entire 

- / ■ 31 ■ ' \ 



interviewing period voXL be cOjv'ured by;the verificatioh process, verification is 
being made at three intervals: two, four, and six we^s after the comnvmpementi 
of the survey. ^ . ' \ . j 1 

Total samples of 60 prdject directdrs, 80 principals, and 120 teachers will \ 
be called for interview verification* Since the purpose of verification is to ensure 
. only that |;here are a very small number of questionable interviews, and not to 
obtain a precise m^'^sure of the number of questionable interviews, the sample 
sizes airi relatively S^mal][.| Adequate representation of all 20 teams has been 
obtained by apportioning the sample evenly across them* One-third of^the total 
number qf each type of verification Vterview will be conducted at each verification 
interval. For each team, the interviews to be verified will be selected by simple 
epandom^'proce(fures from among those that wpre conducted since the last verification. 

TRAINING 01* INTEBVIEWERS . 
Pr.e-Fieldwork Training . ' , ' " * I I 

Team leaders were selected from among riiembers of ^e professional staff of - IT 
'■ the Resource Management Corporation and subsidiaries !or from Mark Bat/le^sociateJ 
a subcontractor. Interviewers were selected iom among graduate stujdenls in the geo- 
graphical areas in which the field work was to be conducted. A total of 20 three- man 
teams were thus estabHshed, each with a team leader and two interviewers. 

All of the teams were assembled from February U to February 19 lol training in 
Washington, D. C. T|ie following paragraphs briefly describe the activities of those 
four days. . \ 

x 

Day 1 . \ ' 

/ 

After the teams checked in aiicj registered, 1^ series of orientation talks were 
presented by RMC and OE personnel. The role of RMC in Conducting the outside 
evaluation of ESAP was discussed. Otj)/r ESAP Evaluations and the needs for results 
such as those that this study would pi?oduce were also coy'erdd. A. history of .school . 




/ 



7 



■/ 
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ion ani 



le^slat|lon and th 
was disfcussed by 
A^rs. Ruby 

4f 



other pderally si^^)port|ed efforts wa^, presented. The current 



\ 



smooth desegregation 



3\rolB that ESAP'is to play in assistir 

1 

B personnel. i > 
ijlakin of the Washington Re^rearch Pr^jec\ described the research 
cond^^ed jointl ^ by ^veral groups^ td^ssess the early implementation of the 
Emei-ke^cy Scho(Ol Ass^tance Progi^am., Her discussion was based on a report by 
thJt consortium entitled Emergency School Assistance Program 

dition, Mrs. Martin imparted ^ex own impressions of the Southern 4'chool systei^.s. 

Day 2 / ' • . . - \ 



In 



Activities on the second day of training included a lepture on '!Th£ Ai^t of Asking 



"Questions. " The lecture described in detail the role and requiteit^ents of a good 
inter^ewer. The following outline show^' the toptes that were covered. 



THE ART OF ASKING QUESTIONS- 
LECTURE OUTLINE 



i ^ 



!• The Role of the Interviewer ^ 
A. General 



1. Collecting Data 
• ^ 2. Objectivity 




/; 



/ 



B. The ESAP Intcrviewefr 



\ 



. ! 



The American Friends Service Committee; Delta Ministry of the National 



Council of Cl^Urchcs; Lawyers CommitteeJj)/civil Rights Under Law; Lawyers 
Constit^Aiondl Defense Committee; NAACP Legal Defense and teducational/35Wd, 
Inc. ; and the Washington Research i?roject. 



/ 
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/ 



C. Interviewing Ettqu4ite ' / 

!• Developing rappiii:--4iformality, human relations 
y 2, Guaranteeing anonj^iiy 
3.\ Ground rules|^on hoVv\o proceed. 

Using th^ iSurvey msti^ent. ' 




(general Structur^ of the pirsonal Interview oiestionnairc 



Preijpoding \j 
2. Bui]|t-in instructions to t^e^interviewer 



B. Specific Structure of the Person^lnterview 'Questionnaire 

!• closed-ended questions * v 

a» closed-ended questions with a' card , ' ' ;^ 
b, , closed-ended questions not including answer categories within 



c. 



the questions ^themselves 
single answers 






d. J multiple ans\yers 

e. nonrresponsiye answers 1 

2. / Semi- open questions 
3/ „ Open-ended questions 

a, the art of probing— specific v^. gen^^l probes 

:• Dependent questions and skip p^^ttems 

Specific Interviewing Procedures 

A. I Conducting the ii\terview, 

1/ Interviewing eti^ctte (i^frain) 
2^ Ar)pointnnronts ' 

3. Settings-privacy 




/• 



j B. Terminating the interview 
C. Validation of ileldwork 

J . ■- 

IV. Points to Remembjer 

A. S wdy related terminolof 

,/ IP 
PS // 



leji 



jB» Ifepkpw anp questions 
After the 



|!ture and the questioh and answer period, a discussion and slide 
presentation of the useU the directoi) questionnaire was presented on a question- 
byquestion basi&'. Quei^ions were permitted during the presentation. 

Following the presentation of the director questionnaire, the attendees were 
sepj^t^d into pairs and each trainee practi^ conducting a cpmplete interview. 
The interviewers were instructed to simulate an actual. Interviewing situation; 
that is, with no discussion of the questionnaire or questioning methodology until 
the interview was completejj. Members of the RMC project staff then-reviewed 
the work of each individual,! in private, pointing out errors, omissions, and other 



prp^ems, and answeringyspecific? questions. 

The group was reassembled and the more salient problems encountered during 
the p]^ctice inter^ewing were discussed. ' • , 



/ 



Day 3, 



^ j The third session of th^ training seminar wis opened with the question-by- 
question presentation of the. student questionnaire and related slides, qnce again, 
questions were pen|iitted' during the Rrcgentatipn. ' '' ^ 



^ Following the discussion of the studeijt questionnaire, "the training was turned 
oveirtpl' consulting psychologist, th^s coni?ultant, a,sp6^ialist in psychodrama, 



ha(3 prepared simulations of situations that] 



;luld 



be exp6ctpd to occur in the course 



\ /'^\ ^^^^^^ ^^^^ vfork. This session was organized s'l^h that, certain RMC staff ^mem' 
V • ^bers play^jdthe roles of tl 




lie various^eoplfe 1^ bp^ncountered in the field. Members 

41 



35, 



of the training groim were instructed to interview an LEA director, then a school 
principal, then a t*ac^er, and finally a student. The trainees did not know what 
to expebt in term;5 of personality typologies presented to the simulation and were 
instructed to c^e with each situation as best they could. Following each simulation, 
the floor wai^ opened to discussion* The role-playing situations offered sensitivity " 
training to th6 interviewer and the entire group had a ch^ce to see how an actual 
interview might proceed under generally adverse co^itions. 

The afternoon session opened* with a lecture devoted to summarizing the acti- 
vities up to that time, bringing the training into perspective, and reviewing 
fundamentals* Following this, the trainees were divided into ^hree discussion 
groups in order"tosttmulat6 an exchange of questions and Ideas* This was • 
followed by further rolerplaying, and the day was closed with a second round Of 
practice interviews and private trainer^traljjiee discussions. 

Day 4 / ^ 

The fourth day of the trailing seminar opened with another round of,pra(btice 
interviews. On this occasioi/, memb^s of the training staff walked about from v 
trainee to o^serving^he interviewi^, offering suggestions, and answering 

questions. / - • 

Following the practice interviewing, a lecture on sampling procedures was 
presented accompanied by a set of practical problems that the trainees solved 
step-by-step with the aid of the lecturer. - ■ 

The team leaders remained at RMC's -offices during^he week of February 22 
for additional training. This training' included further details of sampling, logistics, - 
-^and interviewing, -including details that were to be passed on to their team members. " 
^Chis additional week also afforded the team leaders an opportunity for in-depth 

discussions with members of the training staff. T^^e team leaders were entrusted . •. . 
" with responsibility of last-minute training'of their team members. 
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FIELD INTERVIEWING 

* The Phase I field effort entailed interviews of 252 randomly selected LEAs, 
which require approximately 10,000 individual interviews. The 14-state area 
encompassing these LEAs was subdivided into 20 target areas, within which an 
■ interviewing team was placed. The size of a target area was determined after 
the random sampling LE^s aind was dependent on several factors, including ethnic ,' 
composition, geog'^'apMckl.area, and the number of schools to be interviewed within 
a selected LEA./coasequentljl, the area responsibility of individual teams r;mged 
from one LEA wit&^6 schools in San Antonio to 19 LEAs with 64 schools in North 
"Carolina. 

Tl^e composititife'^^e interviewing team^ varied according to the number of 
schools within a target areK^ak teanuxashg^ed.by an RMC^employee who 
functioned as the team leader. "^1^ remainder>f each team, i.e. , the interviewers, 
was composed of graduate students from a college or university within\the respective 
target areas. The teams were bi-racial and, depending on the number M schools ' 
^ssigne^^^^ranged in si^e from two to«four. Normally, the team leader was respon- 
sible for interfacing wi^ the superintaident and interviewing project directors anc 
principals. The interviewers followed tte team leader into the, LEA interviewing 
teachers and students whb were randomly selected on site, ' t \ 

The Phase I field effort commenced on March 4, 1971. Fox the purpose of this 
report, the field interviews, completed through April 2, 1971, are showk in Table 2-8. 
There were several mino^ instances of difficulty^experienced by certain teams. ' 
The only real problem, ho'^ever, was encountered in the State of Alabama. As a 
Cdrisequence, the field work is requiring approximately 20" percent longer to complete ' 
thaji expected. Table 2-9 portrays the weekly school completion rate by^team and - - 
estimates tl\e respectiyg completion dates for each team': ' 



' • 1.. The only exceptions are in NorthemfLouisiana and Arkansas where thesp 
teams are composed entirely of doctoral candidates from Northeast Louisiana 
University. This exception was made after cdnforring with the DaUas. Regional 
Office and being lold of the political situation regarding ^'outsiders!' in these areas. 
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Special Alabama Situation 

The field" effort in Alabama entailed interviews with 20 Randomly selected" 
LEAs, totaling aBout 81 schools. Four'teams were responsible for completing 
these LEAs, which ranged from the smallest gubgrouping ol two to the. largest of 
nine LEAs. In addition, these teams were responsible for interviewing other 
school system? in states bordering Alabama. 

Besistance on the part of Alabamans State Department of fiducation to permit ^ 
student interviews caused many Interviewing delays and inter-team frustrations. 
Thus, studeilt interviews hkye not (with the exception of Le6 ancj Escambia Coanties) 
progressed smoothly. At best, some students were interviewed after seeurli^ 
. parental permission (a lette;^ orwtelephone caU arranged by the Project Director or 
* Superintendent; at worst. School Boards and/or Superintendents flatly stated ^ 
either ho student Interviews would* be sanctioned 0/ no student interviews would 
be permitted unless a parent and teacheif* sat In during the interview. 

Progress in Alaban.a to date may be divi<ied int6 three categories: LEAs 
offering no resistance for interviewing selected respondent; LEAs requiring parental 
permis^on prior to interviewing students^ ^d LEAs offering heavy' resistance toward 
•student interviews. Table'2-10 categorizes each LEA, lists nunJber of schools pi 
LEA sample, presents LEA completion dates as of April 16 (third column), and 
date of initial resistance (cplumns four and five^ for the; non-pom^leted' school 
system Interviews. - " > • • \ 

A sunimary of the problems associat^e^J with LEAs found in thfe tieavy resistance ' 
column ^re outlined below: " - 

• Baldwin: ,No student interviews unless 

(1) parental 'permisfiion given, and . ' 

(2) parent and teacher miist sit In during Interviewing. ' 

• Colbert: No student Interviews permitted even with parental per- 
. * • ' mtesion. * . ' 
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Table 2-10 



. LEA INTERVIEWING PROGRESS 



LEA 



Numbey of 
Schools 



' , Completed all 
Required Intervicfws 



Students Require 
ental Permfssion 



Heavy 
Resistance 



Anniston City 
.•Baldwin 

Bessemer City 

Birmingham City 
iPalhoun 

Ci)lbert 
; Dale • 

DecaUir City 

Demopblfe? ' 

EBbaCity;^ . 

Elmore 3 

Escambia 
.Lee 

Limestone 
..Montgomery 
Plckins 
'Setna City^ 
Sylacauga City 
Talapoosa 

Talladega City 

A_ 



5 

4 
• 

4 
9 
2 



5 
2 
•1 
1 
$ 
5 
4 
5 
4 
•5 
3 
4 
5 



X (4/2| 

X (4/14) 
X (4/13) 
•3^(4/13) 

X (4/17) 
X (3/31) 

X (4/16) 



Xp/25) 

XC4/16). , 
X(4/9:|-V 

X (4/15) 
X(4/16) . 
X(4/2j ' 



X (3/10) 



X (4/14) 



X (4/7) 



X (3/29) 



X (3/31) 



X(3/25) . 



^\a. LEA experiencing racial problems. Superintendent requests a temporary 
hold before selecting a?id interviewing students, . . ' ; 

b; Students selected and letters mailed to parents. • . ^ " - 

c". Students were not selected. .... 
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• Demopolis^; Demopolls. school board requires . . 

(1) OE or RMC representative must congee- the 
school boaxd of the importaace W interiiHt6^jiiig ♦ 
' student; \ , ^^"^ — \^ 

' (2) If school boar^ls convinced, then g^et parent^ 
permission; ^iKk . ' ' 

(3); a teacher must be present during interview. 

• Limestone; No student interviews permitted. . . . 

When parental permission was sought, a high acceptance rate was e3g)erienced 
(over 95 percent of the LEAs in column 3). Table 2-11 illustrates the. response rate 
per completed LEA. The plan for conducting interviews with study's requiring - 
parental permission is to complete them during the initial LEA visit* if time permits, 
but otherwise to return after all LEAs within the targej;<^^a are visited. Thi?' 
method provides greater team flexibility and ii^cr^ses the team's probability for 
conducting subsequent intefvie\%'s on schedule 

Only two LEAs in other states decli^to allow interviewing of students, and 
both gave permission after discussio^/with OE and/or state officials. 

STATUS OF COLLECTED DATA , - . 

As of ^Aprll 5, a total of 5,155 questionnaires tad been received at RMC. 
These included: - , ^ t 

• 131 Director* interviews . ' ' 

• 470 Principal intei^iews, 

• - 2,470 Teacher interviews, and 
o 2,084 Student interviews. 

A total of 47 LEAs had been completely finished and the questionnaires were logged 
ihatRMC. « ' • * -• ' 
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\ • .Table 2^1i v 

\, _ 

ALABAMA RESPONSi; RATE 

^Anniston City .... . v. ..... ; . .19 Students out pf 2p. 

Bessemer City . .j^j^,-*<^. 20-§t4d^fs out of 20' 

Birminghaia Of^ -..f.. . i} . . . r ♦ » . • 45; Students out of 45 
CaQioun....... :..^^.V... v. ^''i>.» 10 Students outof 10 

Pale. . . . . . . .V".'. .V . . . . ,J. , . ; . 15 Students out of 15 

Decatur Ciiy . .... J. 25 Students out of 25 

Elba City 5 Students out of 5 

Xee '- 18 Students out of 20 

Monl^^nery City 22 Students out of 25 

Kckins 20 Students ouC of 20 

Syl^^uga 15 Students out of 15 

Talapoosa 17 Students out of 20 

.Talladega City , 22 Students out of 25 
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Editing/j'rocedures . • 

/ About 75 percent of all the questionnaires received at RMC in the first two 
weeks of field wOrk we?e edited. At that time a variety of errors were pinpointed. 
There were a series of minor errors, which must be e^^Jected to occur in a^roject 
^ of this size. These errors are correctable" in coding. However, some serious 
■ errors were also disc9vered. Once discovered, the offending teams were notified 
by telephone and meniioranda to cbrirect the errors. ' ^ ' 
Major errors ih(iluded: 

(1) Omission of Information in modules . Four teams w^e found, to 
be omitting necessary information by not asking module questions 
for some $ubactivities, filling out module questiofits incompletely. 
A team inf Florida was sisnt a new Team Leader, and the team In 
Dallas, ^exas, was retrained in interviewing procedures. 

(2) Question lOd in the Project Director questionnaire. Informa:tion 
for Question lOd was either omitted or the percentages for each 
activitjir did not equal 100 percent. Five teams were committing 
this ejttor. 

^ (3) Omission of Project Director activity information in Principal 
and Teacher questionnaires. Attitude questions were omitted 
^ because the subactivlties^that Project Directors gave were not 
. . transferred to Principal ajid Teacher grids. Five teams made . 
' this- error. \ 

(4) Omissions of aided versus spontaneous responses. Some inter- 
viewers neglected to re^cord whether a respondent had mentioned 
the existence of ESAP activities either spontaneously or when 
aided,, though they filled in the respective module. Three teams 
were contacted by telephone and later by memoranda. 

• (5) Module Question'3 describing how materials and personnel are ' 

being used in subactivlties was filled out too briefly. Three teams » 
were aske^ to provide more detail. 

At the time questionnaires are received, random c^ecM. are. made to make sure \ 
the preceding errors have been corrected » 
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' Codmg/-j'rociedureB I 



The first step in codim4he questionnaires was to accomplish a 'tabulation by 
"nand of all the open-ended questions in a random sampler of each'of tiie four ques- 
tionnaires* The following numbersL of questionnaires were hand tabulated; , . 

• '200 Student questionnairbs, ^ 

• 200 Teacher questionnaires, . • 

• 100 Principal questionnaires/ and 

• 50 Director questionnaires. . 

As a result of this tabulation, it was possible to broadly categorize responses 
of a large number of questions into a "conunon code. " These questions include: 

3, 3a, 4a, 4b, 4c, 5a, 5b,. 5c, 6a, 6b, '6c,. 8, 9^ 
. 10a, ^d the subactivity designators in each moduleu 

4, 4a, 5a, '5c, 5d, 6a', 6b, 6c, 6d, 6e, 7a, 7b," 7c, , 
9, 10, ll^L, lib, 11c, and subactivity designators'. 

4, 4a, 5a, 5b, 5c, 5d, 6a, 6b, 6c, 6d, 6e, 7a, 7b; 7p, 
* ^ I ,9, 10, 12a, 12b, 12c, and Subactivity designators. 

Student: • 4," 4a, 5, 8, •12a, i2b, 12c, 12d, 12ei 12f, 12g, 12h, 

-'^^ • 14, 16a, 16bJ 16c, 16d, Ige, 16f. • * . 

Table 2-12 contains the description of the "common, code." The first 19 cate- 
gories relate to ESAP activities while the remaiping 13 are non-ESAP responses. 

,In addition, specially tailored open-ended categories were established for each 
of the other open-ended questions. For the rare instance where questions were the . . 
same in two questionnaires— such as thfe Director and Principal, the same code was 
used for -the same question in each instrument. 

'Complete code books were developed in cooperation between Dataia^),, Inc. of 
New York (the processing subcontractor) and RMC. These code books, as well as 
e^l)lanatory memoranda, are accompanying the questionnaires to the, coding sub- 
contractors. Due to the importance of the Director's, Question 10, it has been " 
coded by^an RMC analyst to assure consistency. ' . 
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" Table 2-12 
.CQMMON CODE 



COMMON CODE 



DPTS 
FOR ALL' 



Indica^^ed 
questions 



1. " PersoD-to-PersoK 

2. Media/Non Personnel 
3» CounseliTig 

4. Counseling Support 

5. Ebhnic Classes and Matorials 

6. Non-Ethnic Classes-land 
^ Materials • / 

7. Training ^ ^ ' 

8. Teacher Aides and^ Other 
Support Persoimel ; 

9. AU^ , 

10. Ail 

11. Personnel (Teachers, 
. Assistants, Specialists) 

12. Programs ,and Materials ^ 

13. Special Comprehensive 
Planning . 

14. ^ Personnel-Administrative, 
all not coded elscwiiere 

Remodeling and Equipmont 

* Portable Classrooins . 

17. All Others 



I 



I. SPECIAL COMMUNITY PROGKAMS 

n. SPECIAL PUPIL-PERSONNEL SERVICES 

m. SPECIAL CURRICULUM REVISION 
PROGRAMS 



IV. ' TEACHER PREPARATI6n programs" 

V. SPECIAL' STUDElA-Td-STUDENT PROGRAMS 

VI. BUSING 

VII. 'COMPENSATORY OR REMEDIAL 
EDUCATION CLASSES 



t<^■'' 



VIII. OTHER ACTIVITIES 
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^able 2-12 (continued) 

X /• 

X y 



18. ESAP CONFLICfT - Personnel : . Use this eode if 1-17 is applieable but is : 

non-ESAP and the sajine 1-17 has been used' for ESAP in Q. 10(D), Q. 11(P), Q. \1 (T) 

19.. ESAP CONFLICT / Materials; Use this eode if 1-17 is applieable ijut is 

non-ESAP and thp same 1-17 has been used for ESAP in Q. 10(D), Q. 11(P), • 
Q--i24T). / ' •• ' 

Racial an'dlnterraeial Attitudes ; All mentions of attitudGs nn mfTn irqupq in 
whatever context. Mentions of non-bcliavioral items such as trust and mistrusk! 
• understanding and misunderstanding, friendliness and hostility, others. ' i ' 

^* Racial and Interracial Deliavior ; ,all mentions in a racial context— fights and \ 
friendly activities, good and bad communications, activities in .all school .. 
settings, all other raca-rclated behavioral mentions- -(EXCEPT CODE 26 

' BELb\V). - . ^ ■ 

22. Intcgration/Dcspgregation ; all references to uniting the races in the school 
and its .facilities. . " . - 

23. Tinle; all references to. time as a factor in creating, increasing, decreasing 
- or eliminating racial problems; ' ' 

24* Academic Mentions ; all references to academic item not classified elsewhere, 
general references to achievement levels and abilities, others. • ♦ ' 

i 

25. Motivationr references to, motivation and motivating forces—student motivation, 
-jnotivation of/by otiicrs. * 

■26. Orgrtnized Social Mentions; sports, dances, parties, clubs, others. 

27« Parents / Community ; all references to parents, community not codable elsewhere. 

28. Atmosphere, Emotions--Non-Racial; all mentions, more freedom, less 
freedom; belter, worse atmosphere; feel at hoitie; others. 

29. Faculty - Teachers. Coaches. Otlicrs ; professional mid social performance, 
role in interpersonal, relations, other ment'ioiis.' • 

30. Students /Class mates ; all mentions. . . ' 

31. ' Special Programs. Materials, ' Facilities; all mentions not codable elsewhere. 

32. . Don't Know/Nothing , . ' " ; ' ■ ' 
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The Director, Principal, and Teacher qu^stionn^res, which all tie in directly, 
. haye all begn subcontracted for coding to Beta Research Corp. of New York. The*, 
first shipment of questionnaires was sent tc/New York on April 12. Three more 
shipments are scheduled to follow « Apigl 16, 21, and 25. The student question- 
, naires are being coded in Washington by RMC. ^ ' , 

Once the questionnaires are edited and coded, they will be^fiady for key- 
punching and cleaning. They are s^eduled to be shipped to Datatab in three'batches 
on Aptil 22 and 29 'and May 6. ' . ^ ' 

Data gyiality Assessment ' ' * 

It is, of^ourse; too early to present an accurate quality assessment of the 
data. Howe^eJ*paftgi^reliminary editing, band tabulations, and preliminary 
coding (Question lO.series of the Director's instrument), some observations can ' - 
be made. , . • * , 

Closed-ended question rules were generally closely adhere;3 to and editing and 
coding procedures will most likely clean up problems that do exists such as minor 
skip pattern violations* • . • 

One of the objectives of Question 10 of the Director questionnaire was to helpv 
develop a better taxonomy of ESAP subactivities. Such a talonomy is under con- 
struction. Table 2-13 is an example -of the items that havJgone into one ESAP 
subactivity classification based on a sample of 117 questiaSnaiEes.. It shows the 
detail by which the ESAP activities are described. ^•^w'^. / • . 

Because of problems mentioned earlier, some data based on tKe' Question lO' 
series have heen. lost. This data base has been further reduced by.the reclassifi- 
cation of subactivjties here at RMC into their "correct" category groups. In these 
reallocation instances, in wliich 15 to 20 percent of the subactivities require shifting, * 
the descriptive, data in the transferred module have been retained but the attitude 
questions have bee ji dropped. This was done becaus.e the-citti.tude questions Were tied 
\ to specific subactivities and thus had been asked erroneously. However, despite 
the problems, re- interviewing, careful' editing, and accurate coding udll help pro- 
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Table 2-13 

TAXONOMY OP ONE OF THE ''COMMON CODE" CATEGORIES 



Code 8~Teachers' Aides and Support Personnel 



Substitute Teachers 
TeaphersJ Aides . ^ 
Communicative Skills Tethers 



: Tutorial Aidea, 
Teacter/Counselor Aides 
Library Aides - 
Mathematics Supervisprs 
Besour.ce Specialist 
Paraprof^jrftJlISTs^ 
SALT Teacher Aides 

r 

Instructional Aides 
Clerical Assistants 
Team Teachers 
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vide a §olid base for analysis. 

The open-ended questions within the modules (descriptive data) often contain 
sparse answers. The quality of these respons^'s-lSjiaried; at? least part of the 
decline in quality from earlier open-edded questions may be attributed to both 
respondent fdtigue and a Jack of the precfse information thes^e questions require. 

Question 5 of the director modules shows very little response; however, 
Qdestion 5 deals with the.reasons for change, in the original plans of different 
activities, thus tile reason for little response may be the fact that few projects * 
have required much change. . < » 



ANALYSIS PLAN 



INTRODUCTION 



>This chapter describes the status of the analy*sis plan for Phase I. ^ ' 
The first section covers the overall approach to the analysis; it discusses why t . 

hav6 selected this approach and outlines the schedule of completion of the ' 
analysis.. The second section discusses the th3?ee major, categories of variables 
used in this analysis— independent^ dependent, and descriptive. The final 
section deals with the special output tables that the analysis will address. Each- 
area is responsive to the broad objectives staffed in Chapter 1. Appendix A to 
this report contaiins exiiinples of the tabulation layouts that are to form the Initial 
Steps in. the analysis plan. ... • 

';• '. - • •' • • • , V . 

CONCEPT OF PLAN ' : 

A two-step pl^n for analysis of the data returning from the field has been 
selected; The first step tries to find the sets pf independent variables that appear 
to explain mo^t of the vari^ce observed in the Values of the dependent variables. 
Consequently, the dqpendent vari^les would t>e dealt with on a more or less aggre- 
gated basj^while a! great variety of independent variables could be studied. " Not ail 
erf the possible dependent yariaMes woidd be incliid^^ 

""^'The results of this analysis would lea<i to' selecting those independent variables \ , 
which will^ be used in the second or final sgt of tabulations. In thts set, all of the 
dependent variables would be cOnsidared in an un-aggrogated form; 
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• By u6ing'thls approach," BMd will be able to produce a yiseM set of tabu^tations " 
broken dcpwn into nieaningfyl categories without havin^b produce every combinatic^ 

: of tables. Ther.e is a ifeitfrfeit of possible independent variables and the logical brekk- 
' downs .of some of them can not be easily prespecified. Thus, Itst'ead of calling.for - . 
certain tables on a on^-time basis, and being limited to those, if is felt that a more ■ 
: msponsive set of. tables will result frpm this two-step approach. 

TYPES OF VARIABLES CONSIDEBED g • ' • ■ .'" 

There are three majn types of variables to be examined in this analytical ^ 
effort. They are the dependent Variables (or measures), the descriptivev • • 

varlablei^ and the independ'^t variables. Briefly, the dependent variables assess 
the outcome or results ^of the ESAP activirtes. These ai'e the prime 
erformance measures. The descriptive variables are used to describe 'the ESAP 
• program and certain other ancillary aspects Such as technical assistance and project. . 
^manageni^t* Tinally,*tbe independent variables are tho^e which serve to explain observed 
differences in tfee two preceding types of variables. The^ generally reflect the Inputs 
of a program or serve to describe the environment in which the program is to operate. 

* With regard 11 the dependent and independent variables, causality is > commonly 
sought. Thus, if one' could attribute certain values of the dependent, variabjes to 
certain values of independent variables.it would be possible to ''cause'* predictable 
lev(Sl^ of jresults ot outcome. * . • , * - - 

This will hot be possible in this study in spite of the. large number of independent • 
variables that are going to be considered. Due to fhe complexity of the , - , 

education process, not to mention the process of integration, there are so many, 
factors that would have to be Included that it would be difficult to try to establish 
cause and effect..; ' ' • 

Therefore, wo will only try'td associate high values of the dependent variables 
with certain combinations of independent variables. Even though some independent 
varlfiblcs williprove ffiTi^e strongly associaljed^th better performance, for 
example,* caiisality'cannat be presumed. ^ , - 



. Finally, 'th'6re are soirieVa'rIables that wiU ser^.e a duaLrole in this^^nalysis. 
One .example is tn the "expectations^' set of questions. They would serve as dependent 
variables when related to the respondent's race' and position in the school, for* 
^ example. Their use as independent, variables would be to assist in ^explaming 
different incidences of peircelved change.. Thus, those with positive expectations * 
might see more-pQsitive change than those of a negative persuasion. In the following 
di-scuiSsioii, certain variables will,appear iii'two i^oles. , 



• Dependent 'Variables 

The r«l6vant dependent variables ara usually easy to specify because they 
haiie been purposefully plac'sd 'into the questionnaire design. .The most obvious and 
Important sets 'of dependent variables arise from the "change" series of questions." 
The first set iiS the mea^ures^relative'to the incidence Of change— -positive or negative. 
l*he otKer is thelikelii^ood that'an ESAP activity is associated by an Individual with the 
positive or negative change; These ar6 the basic output measures. . 

Because of thd nature of the "change"" questions, it is possible to- measure the 
incidence that positive cha^e is observed for several^ facets of schoof operations. 
Academic performance, attendance, and class participation are specifically sought. 
For the out-of-classrooni situation^, measures of personal relationships-- 
student-to^student, teacher-to-teacher," or pai'ental involvement •:7are also sought. 
Each of these variables will be included in the first ^et of tabuX&tioris.. 

In some instances, two will be combined to produce a composite measure- 
attendance is an example. ^ " . 

For each of the above mentions of change, the degree to which ai ESAP activity 
that existed in the school xc^as mentioned in connection with the change will also be • 
treated as a depelident variable. Again, this is a key output or performance measure. 

.As men^;oned parlffer, the "expectations" set of questions are also to be used as 
dependent variables . They can be grouped along a class room/no.n-clasfaixiom basis- 
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as the ''change*' question^ were grouped for jsomparison. For the first step of the 
analytical" plan, we njsed to be able to examine thq way in which these questions were 
answered. Thus, a "positive*^ person might be found to be one who answered three 
or more of the expectation questions 'Very easy." To the extent ^that these expec- 
tations are dependent on the respondent's race, position, or length of time jn the 
school system, these variables must be studied relative to those descriptive pro-^ 
perties. 

In the area of technical assistance, the results of this effort might be a function 
df the DHEW region providing the assistance. Thus, a set o/dependent vayialiles 
assessing the effectiveness of technical assistance is needed. 

*The"5e£s*of attitude" questions asked in the activity-specific modules form the 
last major group of dependent variables! In the first step of the analysis plan, 
they will be treated som6What superficially. Iliere were five possible answers for 
each atttitude qujBStion. These will be aggregated into three categories with any two 
of them being included in the tabulations. The categories will be positive and nega- 
tive in nature without regard to the magnitude of the response. 

"Table 3-1 portrays the important dependent variables or output measures 
to be examined in the initial phasex)f the tabulation process. 

Descriptive Vaf'iables ^ 

Descriptive variables fall into two classes. One class includes ^the descriptors 
of the program or projects^ Tliey serve to show, for example, differences In length 
qf.operation o.r persons employed as a function-of independent variables, but 
there l^ no causality implied or sought. They tend to change oven time or can be 
changed easily. The secQrid set Includes the descriptions of the LEAs, the 
Trespondqnts, and the environment in which the ESAP grant is^ working. These latter 
descriptors are more properly treated §l& Independent or* relatively immutable 
' variables; they will be discussed, more thoroughly in thn next section. 

The -most important set of pr%rtHn descriptors arc those which describe the 
projects anc^ their component acti^tics as imi>lemontcd in Ihc field. Thus, such 
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N Table 3-1 • 

CANDIDATE DEPENDENT VARIABLES 



Type 


1 

Breakdown 


" " i : ^ 

Changes 

• 


Attendance- 

Student-Student '(Classroom) 

Academic Gap 

Class Participation 

Teacher-Teacher . . 

Student-Studenf (Non-Classroom) 

Activities 
* • 

Parents 


ESAP Association 

• 


Attendance • 

Student-Student (Classroom) 
Academic Gap 
Class Participation 
Teacher-Teacher 
Student-Student (Non-Class room> 
Activities ^ ' . 
Palrents * . 


E>5)ectations 


Student-Student (Classroom) 
. Teacher-mudent , . 
Teaching 
Teacher-Teacher 
■Student-Student (Non-Class rooip) 


Technical Assistance 
• 


Extremely and Very Effective In: 

, • ' preparation of grant 

• : swift action 

• operation 

Not Effective Inr * 

• preparatmn of grant 

• swift ao^n 

• operation 
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Breakdown 



Attitudes (fromTVIodules)' 



Project Management 



^ Positive and Negative Feelings Towards 

• " qlassroom behavior.. ' 

• • classroonl performance I ^ 

p' ' community reaction ' * < 

\# ; .'teacher per/O'rmiiice ' :> ;^ • , 

• ' teach^|??student rapport - ^ :V* */ 

By Bace of'Besponderit \ . 



• degree to which, Jie seeks>h^lp^with 
ESAP problems^ • . ■ / 

• result of help ^ . - ' 
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'^alcfeig as* length of operation, Segree.of completion,' numbers of activities funded, . 

• . numlieji^^d ^yi^es of persons employ ed,.^and th^.types^of materials purchased. 

. T)ecome of intd3r6st/ : ] - ' \ 

^. , , Different ^BS^P^ activities woi^^ be expected to use different^mlxes of ' ,/ ' 
personnel and' materials. FundingteyeU might \Ve 11 vary also.. To improve and. ' 

• evaluate ESAP, prdgr^m persomxel must know what resQurdes, in what combinations, 
have been applied to th^' effort, Thiws, tHe dual rple that qau be played by this class . 
of variables fecotafes clear, ProgtAin p^rsdnndl,. for example, .might wish to know 
descriptively how many teachefs^ aides 4^ere utilized By LEAs. in qonducting this • 
activity. Once this is knotvn, t^iiose LEAs.^vith a higher ratio of teachers' aides to 
teachers could be cdmp^Lred wjith*those having'a low r^tib to exanline whether this 
new independent variable appears to be assopiat^ with positive or negative change/ 
Thus, the descriptive variables can always "be Utilized as independent variables 
when needed. The length of time an activity has be^ in operatipn is another example 
of just siich a dual-purpose variable. ' , , * 

Table 3^ portrays the descriptive variables that are to be considered in the 
first analytical phase of this study* They have been chosen to describe the ESAP 
activities in enough detail to enable early description of th? program, as. implemented. 

Independent Variables • ' ^ ^ ' . 



The final set of variables are the independent variables^ Theyjnclude the 
"immutable^' or environmental descriptors sucli as race and &ex of the respondent. 
^ Other descriptors, such as th6 urbanAural Mture of the LEA, and pevceU minority 
' of the LEA, 'or its size, are the variables that are of specific interest to ESAP 
/program personnel. In general, it is desirous tt^at causality be attributed to this 
last sot of variables as program personnel can select areas having the required 
• set of 'descriptors" that wijl produce the greatest impact. ^ *• 

\ . The environmental or immutable variables sci*ve Ic^ temper the observed results. 
Thus, equal dollars per student may have dif cnt impact due to the urban or rural 

I ( . ♦ » » 
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' • ' Table 3-5 

< .♦ , . 

*'..-. . ••• 

CANDIDATE DESCRIPTIVE VARIABLES , 

I ' h' ; ' f — i — [ ^ ;_ 

For Each Major 'Activity Grouping. 

1. Number of activities funded 

2. Number fully implemented , » , 

3, / Nuinl^er partially impleniented 

4, -' Numbernot yet begun * * 

5* Average-length in operation / 

Average doUars/student (a.11 aiad minority) 
7,. Petcent of ESAP funds \ . 
Q. - , Number of blacks employed full time ' ^ 
.9*/ Number of.all others employed fujl time 
,1>0* Number of blacks -employed payt time - ' ^ 
. li* Number of all others employed part time . \ 
12, .Bxistejice of materials purchased * , ^ ^ 
134, Number of students;' aftepted . " ' 
, 14-. Kumber cf schools affected " . 

15* Priticipal^s'knowledge of activity 
^16* :Teacher'fi knowledge of activity 
17 • Number of subactivlties ' ' J ■ , ^ 

' Oh an LEA Basis . * • 



i. ^ Number of. activities . ^ , ^ .. ... 

' ' Z. ■:Mthber bf^sul^tvitW'^'''*^^ 

Proj ect :Manage^ent . » . ' . ■ 

■ !• Degree of involvement in planning of stipei*intendetit 

■ 2. Degree of involvement in plamiing of director 

3. Degree of involvement in plaming of principal 

4. Degree of involvement in planning of respondent 

5. Degree of involvement in implementation of respondent 
. 6. Incidenqe 6f problems ^ - . . - 

Technical Assistance 



. It. Degree to which o^ch activity g)-ouping received- 
technical assistance 



•Table 3-2 (continued) 



Verification ^ 

^ !• Degree to which extant plan is same as approved plan 

2* . Pegre.e to which director and principal agree 

3 • Degree, tb which director teacher '^ree . . 

4* Degree to which principal and teacher ^ree 

Responddit Data 

1. Age 

2. Education level " • 

3* . Years of education' experience 
4* Years in local educational syst^em 



65 



ERIC . 



59 



nature of an LEA. .Blacks and non-Blacks might see the same results In a different 
light. Thus, these variables must be examined to ascertain which are the most 
, important. 

These variables have important policy implications, however, as1)riorities 
must be set to assist in the fund allocatipn decisions. Thus, if it can be shown 
that small urban LEAs with a high ratlQ of no?i-minority students appear to be using - 
the ESAP funds to best advantage, tiiese types of LEAs might w^ll be funded first. 

■ Elementary schools might be funded before secondary schools if the analysis 

• , • ■ ■ " r 

would substantiate this. '. ' , ' . ' 

' ' ' ' ■• . ' 

The so-called impact variables, however, are the ones mqst under the control 
of ESAP program personnel. -They can vary the amount of money spent in any LEA, . 
the types of allowable activities, and the. mix of personnei to materials to 
be utUized in implementing thd activity. Although length of time in operatioii is not 
directly under their control, they need to,kfaow-the results over time td be .able to 
assess quick response activities as opposed to more Ipnger range activities. Finally, 
* the provision or absence of technical assistance may weltaffect outcome. 

Table 3-2 lists the independent variables that, will be considered in th^ 
initial analysis phase: They are broken down into' descriptive and policy. types. .of 
variables. , . . < 

Schedule . ■- . 



■ It Is planned that the first set of tabulations will be available in mid-May. In 
addition to the tabulations, listing of all responses will be produced so the ranges 
of the certain i^idependent variables can be specified* Following analysis of this 
flrstsetoftabl.es, specification^ for the final set will be prepared. Since the 
data tape from which the tables are to be prepared will exist in a complete and 
cleaned version,, these latter tables cAn be produced with a r^tiYely short, turn- 
around— a week" after specification being the longest inter'jf^J^ ""^ 
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Table 3-3 



' • CAiqDE)ATE INDEPENDENT VARIABLES • 



Descriptive . , 

' 1. Respondent's position 

2. Respondent's expectations 

3. Respondent's race » 

4. Rac6 of respondent's superior ^ ^ 

5. Respondent's experience 

6. LEA urban/rtijpm designator, * 

7. LEA pje'rcent minority 

8. .'School percent minority 

9. LEA percent students reassigned 
10* Sdhool percent students reassigned - 
!!• School size 

12* School type • .\ 

13 • 'LEA size 

14. DHfeW region * ^' . 

15. Statue • ' • ^ 

16. Length of Time School has' been integrated 

Policy • , " 

1'-^ Types of activities funded 

2. Types of subactivities funded 

3* Funding levels ($/student) 

4* Percent of all funds in activity that are 'ESAP 

5. Length in operation 

6. - Existence of activity in school 

T. Personnel/Material mix * 

8. Types of personnel employed 

9. Provision of technical assistance 



PREPARATION OF TABLES ON SPECIFIC AREAS OF INTEREST TO OFFICE 
OF EDUCATION - . - . , " 

' ' ■ ■ ' ' ' 0 

% * - • . 

The preceding section discussed the three sets of variables that are t^ be 
considered in this pfhase of the analysis plan. Jn^this section the way in which tl;ie 
variables are to be related or associated to meet specific information needs is 
covpVed. li^each combination of independent ancf dependent variable welre examined, - 
however, the number of tables produced would be far too many, Cpnsequently, a • 
plan to produce a modest number—about 70~has been developed. . 

Verifipation and Description , ^ < 

^ ' To meet these needs, the descriptive variables will be used. For verifica- 
tlon, we plan to use the degree of matching between wVt the dixectpr _ 
says and the grant, principal, teacher, and (later) BRAC responses. These variables 
will be related to LEA siz^, activity categories, school type, the race of director ^ 
relative to the percentage minority in the LEA, and the existence'of technical.assis- . 
tance, ' ' - , , 

For the program description section, all -of the ofhei: descriptive yariables such 
as numbers funde'd, length of operation,, schools affected, etc. vpH be related to LEA 
size, activity types, urban/rural designator, type of school, percent minority, and ' ' 
•percent reassigned. 

Poli6y - / • ' - . 

This area wtll be Addressed in two parts. ^For both, however, the same set of 
dependent variables will be used. These are the ''change'' variables shown m Table 3-1. 
The first set dbals with selecting districts" or LEAs for funding. Thifs,' the appropriate 
independent variables would include the LEA descriptors such as urban, rural, LEA ' 
size, percent minority, percent reassigned, and the like. * 

The second set concerns the activity selection. In this exercise 'the dependent 
variables will be related to the types of activities going on in the school, the funding 

^ • : ' • 
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Jevel of the activity, and the length of operation of the activity. In addition, both 
clai^Ses of tables will be tempered by the race, position, and expectations of the * 
respondent, to properly assess their impact. , ' ' • 

Project Management ' • • 

This area uses both dependent and descriptive variables. The dependent variables 
are those which have to do with the incidence of difficulty withihe operation of ESA3P , . 
activities and the seeking of assistance to help solve the problem. Also, the results 
of this assistance ar6 included. On the descriptive side, the persons who were involved 
in the planning and implementation process will be tallied. The relevant independent 
'.variables are the race qf the respondent, the race of his superior, and his position. 

To some extent the knowledge of ESAP activities has a management impact as 
well as that of verification. Thxis, the same set of variables used there would be 
associated with the above-mentioned independent variables. 

Technical Ass istance * - 
7- ; «^ 

The stubs for this set of tables will include the activities ^that received tech- 
nical assistance and. the results of that assistance. The independent variables would 
be the DHEW tefgion and LEA size. 

. Personal- Data ' " . , . " ^> 

Two kinds of tables comprise this section. The first is to relate the respon- 
dent' s position and experience to his age, race, and sex. The second is to pategorize 
the expectations of the' respondents by the same characteristic?, including their 
' experience~in both case^, the experience in the LEA a? well as in the field 
of education. , * * . 

Sample Table Layouts 

Appendix A has been appended to this report to show grapliically the relationships 
discussed above. Tlicy arc laid out in the sanie order described above and they 
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comprisfe the cross-tabulat\ons that will be .prodv|ced in the Initial analytical run. In 
addition, there will be several.tables produced that will port«:ay the results to certain 
questions without any cross-tabulatibn or categorization^ These include a list— or 
taxonomy a^ it were— of the ESAP subactivities, the numbed of persons employed in 
the ESAP projects, aggregate answers to the attitude and expectation questions,, and 
the incidence of change in the selected areas. 
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PHASE n PROGRESS 'EXPORT 

As discussed In Chapter 1, Phase II of the ESAP evaluation is an In-depth study of 
20 local educational agencies whose projects have been meeting with success and 
can be described with some reservations as "exemplary" districts. In this chapter 
we discuss the progress that has been accomplished toward meeting the objectives 
of Phase n. ^ke topics presented In this chapter Include: (1) a brief synopsis of \ 
crltetla and proceQures for selecting the 20 school districts; (2) the guidelines, 
objectives, and accomplishments of th^ orientation visits; (3) the approach to the 
Phase n analysis; (4) the evaluation design; (5) initial field work; and (6) the 
preparation of survey Instruments^ The schedule for past and remaining Phase n 
tasks Is, summarized in Table 4-1 below. *v 

It should be pointed out that a significant part of the work in Phase n is belnjg 
conducted under subcontract by Mark Rattle Associates of Washington, C. *MBA 
is responsible for part of the field work and on-site evaluation effort of Phase II 
as well as for contributing to th6 stu^y design, aiialysis, and documentation, 

' SAMPLE SELECTION • 

To assemble the list of local educational agencies fot Phase II evaluation, RMC 
solicited nominations from a variety of agencies who had current knowledge of the 
events ^nd progress at dlslricts^wlth ESAf projefcts. Recommendations were ob- 
tained from: 

(1) . HEW Regional OfCices In Phlladelpl^a, Atlanta, Dallas, and 
Chicago, , ' 
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• ; V. : • V* , Table 4-1 " 
'•V 

. • '\ SUMMARY SCHEDULE FOR ESAP PHASE II 



February 26 


Complete sariiple -selection and set up initlai visits to 
26 LEAs. 


March 8 


• • 

• * . 

Begin orientation visits (6'two*-man teams) . 


March 22 


Reduce sample to 20' LEAs and deVeJcp comprehensive • 
plan for^evaluatidn and data survey Instruments. 


April^S^" 


Begin site visits for*unstructui*6d,(non-:survey) data. 


' AprU 19 


Submit survey Injstruments on 0MB approval. 


May 3 


Begin site visits using survey instruments. \^ 


June' 11 


Begin documentation of-20 case study reports. * 


. July 15 


Submit Phase II Draft Report to Office of Education. • 


August 15 " 


Submit ESAP Draft Report to Oflfce of Education. 
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' . (2) University School Desegregation Assistance Centers In *. ' 
. twelve Southern States; ' 
* * *» * * *, 

^ (3) * Washlngtoh:Res'earch Project— a ^non-profit organization ' 
" "f. doing darly ESAP evaluations; 

• • (4) .OE Division of Equal Educational Opportunftiesj • ; . ^ . 

, <5) Staff of Senate Select Subcommittee on Equal Educational t 
. Opportunity? and ' \ 

(6) HEW, Off ice of Civil Rights. • ♦s^' 

' *" * ^ * • » . • 

The ^eclston ws^s tnade to select 26 candidate gltes Initially and visit all of them 

; befpre making the final selection of 20 LEAs.. The foUqwing guidelines were 

established and s^t forth for selecting ESAP projects foV Phase n evaluation- 
; • . ' ; / : \ \ / ^ ' 

0 • (1)' ' Select projects that aye hej^lng, school desegregation in a ! * 
particsularly .effective or'ihnoyative way; . ,i 

• (2) ' Project shoifilji be oif resiason^le si^e In terlns of the amount 

. of: resources 'efval,la^le to the LEA pa^gficipants. .•Ayi>id prat 
. • jects .where' resources are thinly spread* wer sphool district* 

. (3) Project should b$ reasonabfy well along^in Its schedule prior 

. . ..'to^our -Evaluation* . 

• ' • ' • . ' ' . . ; 

• , (4)' Project should not be already existing program. It is okay . 

. •. i'f project was in planning stdges 'pribr-tb ESAP funding. ^ 
I. * ' . ' . ' ' ' 

Any reporting by loqal newspaper highlighting positive aspects 
I of ESAP project would be useful. 

(6) Likelihood of LEA cooperation should be 'good. ^ 

* (7) Avoid districts that have been extensively reviewed by bWier 

outside agencies. 

. '^By mid- February a master' list with approximately 75 LEA sites was compiled 
from tiie responses from all four sources. These were grouped by state and checked 
With reports and personnel at the Office of Civil Rights for racial balance and com- 
pliance to court order and HE^ desegregation plans. Districts with .questionable 
records wore eliminated. ^ * % ' 
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An investigation was then conducted of the remaining LEA s, with ESAP grant 
applications as a reference for project design and funding. The sample was reduced 
to 26 LEAs, balanced^ according to size and variety of projects. Table 4-2 presents 
several of their relevant characteristics. 

The sample was finally reduced to the desired number of 20 upon completion 
of the orientation visits. Cuts were made on the.basls of the subjective opinions 
of each field team, considering a variety of factors which Inclu^ded size of the ESAP 
grant, variety pf projects, and geographic location. 

■ . r ■ 

ORIENTATION VISITS ^ 

For two weeks In March, six field teams visited 24 LEAs as listed in Table 4-3. 
For different reasons, trips could not be arranged to two LEAs—Prince Georges 
County, Maryland, and Montgomery County, Alabama— and these were dropped 
from our sample. 

Seven primary goals and objectives were defined for the orientation visits. 
These goals are listed below along with some of the guidelines to the field teams: 

(1) To establish rapport and a climate of acceptance and cooperation 
for the in-depth evaluation: 

• Compliment the superintendent and others at the 
orientation meeting pn the fact that the prpgram of 
the District, under ESAP, has been Identified as ^ 
exemplary. ' • ^ ' 

• Establish that the purpose of the evaluation is to find 
out ^Vhat It Is that Is being done that makes it effective, 
for use by other districts. A second purpose is to 
elicit what more n^eds doing that might be considered 
for future funding. A third purpose is to use the visi- 
tation staff expertise to help Iniprove the progranri if 

.desired. A fourth purpose la to* provide as much data v 
as can Be gathered assist Congress to make future 
funding decisions. The pui^ose is not to be critical. 
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\ Table 4-3 . 

\ • * 

PHASE ILNSCHEDULE FOR ORIENTATION VISITS 
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' Team A 

Dorchester, Maryland < 

Sumter, South Carolina 
West Palm Beach, Florida 
Polk County, Florida 


• 

Phase n \ 

March 8 
.jYiarcn tJ-ii 
March 12-15 . 
March 16-17 
March 18-19^ 




leam v 






^ * Tuscaloosa. Alabama { 
Macon County, Alabama 
Auburn City, Alabama \ 


March 11 
Marph 12 




Team C 






Dumas, Arkansas 
fiidianola, Mississippi . 
Williarnson TpnnpQQPo 
Kankakee, Illinois 


. March 8-9 
March 10 
jviarcn ii— ii6 
.March 15-16 


1 


* ' Team D 






' Durham, North Carolina 
Pittsylvania, Virginia 
liBXinorton. North n^jTnlinn 
Salisbury, North CaroHna 
Caiarlotte, North- Carolina 


March 9-10 
March 11-12 • 
jviarcn lo 
March 16 ' ^ 
March 17-19 


t « 


Team E 




- 


Escambia County, Florida 
St. Landry, Louisiana 
La Fourche, Louisiana 
Harrisburg, Pennsylvania 


March 8-9 
March 10-11 
• MaVch 12 
March 16-17 




, team F 






A, San Antonio, Texas 
/ Abilene, Texas 
/ Hampton City, Virginia 

■M — _ _ 


March 8-9 
March 10-11 
March 17-if 
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(2) To secure accurate, up-to-date information describing the . 
programs financed with ESAR funds. To'Identlfy differences 
between the program as operative and the proposal submitted 
for funding. To id^tjjfy;^the rationale fOr-the differences; 

• where appropriate, Visit* the program In action 
and talk to ^elected pa^tfcipants to determine if 
their uiilerstandlirg of the program coincides or ^ . 
* dlJEferjs with tbp' program description; and • 

\ ' # ' ;Compar(B,the program description '^vlth.the pro- - \ . , 
/ :. ^y 'lio&ar submitted' for funding and- elicit and record 
; , . ' vtijiieS' rationale for the major differences^ 

'(3)""' Tq/ldBjifeify*knd\vded^ personfe'wlthin the school system, 
* conaii;inlty, unlver^sities/ etc.; whose. opinions might'contrib- 
ttteUo'a Tsicaind evaluation: * - * ^ * 

•;:l^jSave:fh^.su^ei>lntend€nt iind his associates nomi- 
•> 'v' : such people. Secui^e position tUie§, addresses^ - 

*i ; and.telephone numbers. . Secure the asslstknc^ of the ^ 
.-yll^it * * Jsuperm^eadent in an^anging intmlews.and a sichedule, 
V lf;apRitopriate..v - ' ' ' ^ V ' • * 

W/To eticit.?nputS"fFDm, tf^^ superintendent aiid 9tbte-r p^^rsons at;the 
/ ; orieiitatiQn visit, '.a^'io ^hat they.^ considf^r Vi^ adequate ^. 




/; what-tkej.t]^^' ^fc happening* \ ^; 

/ • ! "^^^-^^^^'^ ^^^y sUpportirig;d^ta'^a^ infomatloh.' 

^ already' on hand/. ahH\feecur^^^ it.* -tf-not on'band, dfscuss ' 
'•\ J* . Wa3rs'bf obteihlng"!^^^^^ / .V"'/. 'T' ' C- 

j(5) To'lnsure tlbat the total of .people' to be'cpntactiBd during the evalu 
\ . atioh Is repfesTsntative Df the mi)iprity and majortty population: - 

/.y /Thi^.probabl^ cannot o- • 

p'rtfporlldn <tf mniorlty .bo majorl^^^^ ' 
■ " ' Judgment will have to be the .Criterion. / . 
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. (6) To secure a reasonably comprehensive knowledge of what 
kin^s of information alreaS^ exist in the school and agency 
records that might be helpful in the evaluative process: 

• examine available central office records on . 
students, teachers, attendance, drop-outs,. 

' etc. ; and ./^ 

• record current and past prevalencfe <>f racial 
.5' conflict, racial'cooperation and participatlbh . ^ 

, in school and community events. ' - .. 

(7) Begin to develop a custom-design for the evaluative process 
• of the District's Program: . ^ 

• begin to work out.a synthesis qf RMC's Initial 

design, and all* of the above inputs to determine ' ^ . * 

what is reasonable and possible to measure; and 
^* to the extent possible—how. to. measure them. 

' • Fpr the most part the c*jectives and goals of the orientation visits were accom-- 
plished; Sije reports were prepared and a summary chart describing each ESA? 
•project activity was assembled to aid development of a comprehensive evaluation 
plan. The resul;ts of this work are desc^r^ed in the 'fallowing sections, 

APPROACH to" PliASEH ' ".• * ' ■ . 

t « 
«• • , • 

The !phase n evaluation of ESAP funded projects at 20 local educational agencies 
has been separated into t\vb parts: . " • 

(1) individual case studies; and \ - 

(2) Qomparative analysis of ESAP' activities. 

, * • 

, ^ The first research effort will be condupted by six case-^udy t^drtis--each 
responsible for three or four LEAs. Each team will be responsililc for 
work and data collection activities according to the guidelines set foi?di in this 
evaluation plan. The.feams will prepare ijase- study reports in a Xlume or series 
of reports separate from the other analysis and Independent of t/io PhJise I report. 
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The other parts of Phase II evaluation will be completed by a smaller group o 
analysts using dafia collected by the case stadff teams and their repolrted findings 
and conclusions; This work will be conducted simultaneously with much of the 
Phase I analysis, and will be Integrated as much as possible with that effort. 

^he two parts of Phase II evaluation are specifically designed to, address 
different aspects of the ESAP evaluatloli and will answer, Separate questions. The. 
LEA case study reports will focus on these questions: ' 

(1) What are th^ special problems of each LEA? * V 

(2) What are the ESAP project objectives? Are they cl^rly stated?*"'*^'"^ 
<3) . Da objectives relate to needs of t.arget population? ^ ^ 

(4) How do goals relate^to LEA desegregation? . Education? . ^ • , 

(5) What were features of project plan? ' • * ^ ^ * 

(6) Did LEA plan relate to objectives? * ' : 

(7) Is project making progress toward goals? , ' , 
J (8) What is link between plan and actual LEA activities ?^ 

(9) . Is project staff good? Trained? 

(10) What are results^ ; ^ " 

(11) Is project efficient In terms of output achieved for glv^n Inputs? 

(12) Are res\xlts meeting objectives*? 

(13) "^ow do results relate to desegregation? Educational quality? 

TJie second part of the Phase II evaluation will be an analysis of similar fiSAP 
activities (e.g., community programs, pupil personnel services, teacher preparation 
etc.) at tj^he 20 LEAs. By examining the yarlety of activities at a number of LEAa, 
common trends arid rclation^ships \^11 be sought. , These questions will be addressed: 

(1)' Does each ESAP activity adclress a commoit proiblem'^r set 
oif problems at the LEAs? 

V'.?»{2)'TIs^tTiefe a"cdmnii^^ . ' , 

" ^3) How do local needs and objectives affect activity designs? 

.(4),. What foctors Inttuence actual progiM3ss and pro6tedurcs ? • 
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V (5) Wha£ influences the efficiency of ah LEA activity? ^ ^ 

(6) What activity results can be expected? 

"(7) Can factors influenojing success or .failure isolatisd? " ' ' ' 

»*'*>'*, 

(8) Are program results related to desegregation? * 

(9) What are educational benefits ? • 

In addiction to separate activity analyses^ a number of pi^^j^questions will be 
paised and examinations made to compare the relative impacts of different ESAP 
activities ♦ These issues will be evaluated: 

r * , \ • 

(1) Which ESAP activities* hay e objectives strongly related 

to desegregation? \ , 

(2) Which activities focus on educational objectives? ^ 

. (3) What is relative ranking of activitie3 with respect to 
rol^ n desegregation? Education? 

' (4) Are certain activities difficult to implement? 

(5) Which ESAP activities offer best results in short run? 
/ In long run? 

(6) ^ich activities are most efficient in terms of actual 
results achieved for inputs used? ' ^ 

EVALUATION DESIGN • , 

To accomplish the objectives set forth in the Ph^isg 11 evaluation and inswer the 
long list of questions and policy issues, a comprehensive plan for the overall evaluation 
design has been prepared. The plan establishes the framework foK both Phase 11 
tasks: the individual, case study evaluations and the comparative evaluation of ESAP 
act'vities^ In adrfitlon^' it gives structure to th^ identification of data needs and the 
^preparation of survey Instr*uments and data collection forms. 

General Structure • " ' . ' ^ ^ 

The overall evaluation design for Phase II is summarized in Table 4-4, It is 
structured in four parts relating to the chronology of- events surrounding a project's 
inception, design, implementation, and impact. The four categories are defined , 
as follows: 
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Context : Descript^D)i of existing local conditions and environment 
that stimulated project design. Includes Identification of special 
heeds and problems as well as target population. Most Important, 
context defines specific objectives, goals and expectations of the 
ESAP project. ^ ' 

Platitie{j[^ Design : /Despribes the original project plan designed by 
LEAlto^^leve the ESAP goals. Includes statement of procedures, 
chedules, budget, staffing, facilities and role of participants, 
Plans can be n^odlfled to meet revised objectives or planned 
variations for specific schools or participants. 

. Actual Process ; A complete reporting of ESAP activities asAhey 
' * actually occurred wAh a full reporting of air events and c^umstanaes 
that 'affected the process. Includes when and whereaetmties were 
conducted, who participated, how long it la[sted<Snd other relevant 
facts. ' \ ife 

Output: T^is category defines th6 ESAP project results in terms' 
of specific effects or impacts. This could include changes to: , 
'existing organizations or structures, attitudes and behavior, and 
academic performance. Unintentional or unexpected effects 
shduld also be noted. , ^ 

As suggested in Table 4-4, Information and content identified in the four parts 
of a prw)ject will be sedured in three ways: 

Claims refer to reports by Individuals atftj^t events in which they 
wepe directly involved. TTheir Involvement can be as project 
designer, administrator, participant, or anyone ^^^o^las a special , 
' ^ interest In the project^s effect* These data can be qbtained by 
per sonaljlnterv lews, written questionnaires, or mailed question- 
naires. In all cafees, however, the- responses are Individual 
opinions. We must realize th^t even though these reports may 
in some cases be unreliable and misleading, they constitute the 
primary source of data ft)r the ESAP evaluation. 

' Verification is the attempt t^ obtain outside reports and direct 
evidence to substanljiate the claimed repoi:ts. '^Thls applies to 
^- claims about project context, design,, process, and outputs* 
Jn some cases opinions of persons outside the project are suf- 
. ficienl evidence^ for Verlficatipri. In other cases written reports, 
, memos, obsorvalions, and other ''hard*' data should be seciured. 
The most difficult of all tasks in the Phase II evaluation plan will 
Ije the verifiealion of project outputs. This catj be traced to a- 



number of reasons. For one, the lack of pre-test/post-te?t 
data; for another, the limited time perlbd which most ESAP \ 
1^ projects have been operating. Many .projects—particularly \ 
curriculum revision activities— are still bfeing developed and 
will not be completed until next year. 

^Evaluation , as Identified in Table 4-4, requires a value judgment 
to^be made about the worth of the reported (and verified) context, ^ 
design, process, and output^ofallESAP project. Specific evalu- 
ation topics are listed in Table 4-4 for each of these areas. 
Judgments can be secured frpm ESAP staff as a self- evaluation. 
* However, the primary responsibility will fall on RMC and MBA 

evaluators. ^ . * . < 

Guide to Data Collection 

The evaluation design for Phase 11 provides a convenient guide and check list 
•for defining data collection needs. Detailed lists will be prepared for each ESAP 
activity group identifying what Information is needed for each cell in the evaluatioa 
matrix. The need for data beyond just self reports is clearly evident and will haye 
to be secured from a variety of ^pources. The need for pcoxy measures to test and 
verify project effects is essential to complete this cell for many projects. A more 
detailed discussion of data collection needs is given later. » 

Framework for Case Study Evaluation 

The basic structure presented by the matrix in Table 4-4 gives a complete 
frame\vx)rk for conducting the Individual case study evaluations of ESAP projects. 
As noted In Table 4-4, evaluations can be made about project context, design, 
process, and outputs. The main analysis should center on how outputs relate to 
context and how actual process related to design. 

Written documentation about the contents of ^ach cell in the matrix and a dis- 
cusslon of the relationships between cells for each ESAP project will constitute a 
complete and thorough case study report for each LEA. 
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Comparative Analysis of ESAP Activities 

Again^, the same matrix structure used to, guide the data collection and individual 
case studies will be the key to our analysis of the major JSSAP project activities. 
Analyses will be designed to compare the contents of identical cells among districts 
with similar projects. Causes for differences (or similarities) will fee examined 
for project context, design, process, and output. Techniques such ds 'factor 
analysis, correlation and covarlance analysis, and multiple regression techniques 
are possible tools for Investigating and performli^ the progtam analysis.. 

Having completed the analysis of Individual ESAP activities for the 2T) LEAs, 
It Is possible to attempt a comparative evaluation of their relative i^erlts. The 
success of this endeavor will depend In part on the outcome of the previous analysis, 
I.e. , the abUlty to measure and detect differences In project context, design, 
process, and output. A weighting schenjie will have to be devised If a simple 
ranking of ESAP activities Is; to be accomplished. More likely, however, this ^ 
evaluation will lead to activity comparisons against multiple criteria and no single 
Index or measure wlli be derived. • • ** . 

DATA COLI^E^foN NEEDS 

I 

As pointed out^prevlously, the evalueCtlon matrix design will be the basis for . 
collecting data for the Individual case study evaluations and the^subsequent cross- 
dterfrlct analj^ of ESAP activities. The data needs conforming to E^AP project 
evaluation are? discussed In the next section, followed by d list of general LEA 
data that will be collected and a brief description of the data collection guides and 
. survey Instruments that wlll^be^uaed* ^ 

' ESAP Project Data ' ' \ ^ • 

Data will be collected for each ESAP project activityttt the 20 LEAs in the 
Phase II sample and will follow the format prcscnlt d in Table 4-4. A general 
data guide will be prepared for this purpose foi- ouch projecftypc and for each ! 
LEA. Certain select districts with .espccidlly'lru^u^/i or different ESA;%projccts 
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wUl be examined In more depth than others. This means that more persons will 
be contacted and more time spent collecting direct evidence. * 

Projects that are very small will recelv^e proportionately less field time for 
data collection than larger projects— especially if they co-exist with larger ESAP 
projects. For some of these it Is possible that only "claims and self report" data 
will be collected and no verification or self- evaluation effort made. And for 
especially small projects, we may disregard thehi completely In the LEA case 
study. These decisions will be made as the field work Is Ith^mented, 

General LEA Data \ • 

There are a number of data categories, If examined, that are Indicators of 
trends in school districts. For example. If the average pupil dally attendance In 

a school district Is 89 percent for the school year 1969-70, and Is 94 percent for 
the school year 1970-71, this Is a healthy trend. Something must have happened, 
or is happening, that brought about this Improvement. Chances are, there are a ^ 
number of things happening. Chances are equally good that no one "happening" can 
be Identified^ with certainty, as being the cause,. However, If nothing has been 
changed In the school district except that one "happening, " one might infer, again 
Utth qualifications, that there Is a relationship between the "happening" and the 
Improved attendance. 

It Is for the purpose of Identifying significant trends, or the absence thereof, 
that general data should be secured from all of the districts Included In Phase II 
of the ESAP evaluation. ■ Using the attendance example again, if 18 of 20 districts 
show marked Improvement In attendance following the establishment of ESAP 
funded projects, the statement can be made thai It appears" districts with ESAP 
projects are improving their attendance averages, even though one cannot ascribe 
the improvement positively to ESAP projects. • " 

' There are also incidental factorsihat sometimes can be found In some of' the 
data. For example,, an examination "of school budgets and staff deployliu-nt data 
will reveal quickly whether or not a counselor being funded by ESAP is in a \lltion 
to or a replacement of someone included elsewhere in the budget at some tiinc. 
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To accomplish ^valuation of a program for v^icl/hard data are not readily 
available^ It Is sometimes useful to Identify ^ren dg^ Another example, If 18 of 
^ 20 school d^trtcts show significant decreases in incidence otvandalism following 
the esjablishment of a prograri^ or programs not necessarily directed toward 
decreasing vandalism, thte trfend can be identified and pointed out. If something 
of this kind can 'be sho\vn to be happening aftrer ESAP, even though it cannot be 
cotfcludjed^that ESAP*is doing it^, the cointiidence of timing o^be suggested. This 
is of importance in educational evaluations because educational resultl are so 
often the product of a number of contributing factors, the contribution to the result 
by^ single.factor being difficult. If ijot impossible, -to identify. 

DATA COLLECTION GUIDES ^ ^ * 

Data collection guides vdU be prepared for securing both specific Reject data 
and the general LEA data, The data collectioh guide for project information will' 

• follow the general Phase II evaluation matrix structuV-e. Separate ^ides will be / • 
used for ctAtext, design, process, and output information for each ESAP project*" 

"^S5!ivify^(i.e. community information,, special pupil personnel services, etc.). 
The -guide will identify what jiata are to be collected and the recommended sources ^ 
and persons to be contacted in order of importance.i^The guide for* context, design, 
and process data wi/l be identical^ fbr all ESAP activities. The data guide for 
project output information will vary by tj^pe activity., The list of persons to h/j^ 
contacted for all ESAP project data collection guides will be designed specifically 
fou each project. . * *- ^ 

SURVRy INSTRUMENTS " ' ■. . ' ' 

.Three structured survey instruments will be needed to secjiirc common measures 
of ESAI? activity content, design, proqess, and d'utpat across all districts \vi\h similar 
-aclhit^. The instruments ai^e designed primarily to secure uniform information 
anU rci.ponses in a consistent manner from iiU LEAs in thr .Phase II sample'and will 
ijC"Us('d in bol!> Iho ease st^idy evaluations anti the cro.s:> di tiicl analysis. A separate 
inslrumenl i^ boing used foi*: v» ' ^ _ ' ' 



■ V ' 

• parents, 

• cdUnselors, and 

• teach eijs. 



Table 4-5 summarizes the areas, of inquiry covered with each in:^trurnent and 
the technique to be used. The areas of inquiry are keyed to the evaluatiofi matrix 
design presented earlier. 
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Table 4-5 

ESAip' PHASE ir SURVEY INSTRUMENTS 



^ Target 
, Group 



ESAP 
Program 



Area of - 
Inquiry 



Technique 



Parents 



Community 
Information and 
Commmiity 
Understanding . 



Claims : process, output 
Verification: context * 



Mailed 



Counselors 



Pupil PersD'miel 
Service- 

and General ESAP 



Claims : design, process 

output 
Verification:' context^ 

design, process 
Evaluation : context, plan 

process, output 



Interview 



Teachers 



Attitude Training 



Basic Skills 



Teacher Aides 



Curricuhun Revision 



Claims : process, output 
Verification : context, 

process ' 
^Evaluation : context, 

process, output 

Claims : process, output 
Verification : context, 

process ^ 
Evaluation : context, 

process 

Claims: process , output 
Verification:' context, 

process * - * 
Evaluation : , contexi:, 

process 

Claims : process, output/ 
Vcrificaljon : contexi, , 
process ^ / ;* 
E\^aIuafioh : contcxi:,\^ 



Mail back 
Module 1 



Module 2 

/ 



Module 3 
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APPENDIX A 



' •• ^ tABLE DESCRIPTION^ , ," 

} • ' . . 

'pkis section briefly describes nine tables chpsen from a set of about seventy cLe- . 
signed to be us^d in the first phase of the analysis'" These nine were.chosen->only 
as representative samples. Xhe set of tables are divided into' eight areas of interest; 
only one or two from each ar6a are described in this section. -* The eight ar^as are 
listed below. 

• aI^RITICATION (Table 64) . , 

•> Table 64 presents the incidence that LEA personnel -are knowledgeable of the 
ESAP subactivities in an LEA or school as a function of the percent minority and 
^ ^ numberof slaidents in the LEA. Matches are recorded when a director's response" • 
matches that on the grant application, when a" principal's i^esponse matches the 
director's response, and when the teacher's response matches the .director's response. 
This tabk will give us an indication 3t how many of the respondents were aware of 
the ESAP activities and also give us some insight into how accurate psponses were. 
These results can then be compared to program effectiveness. Other tables In this 
^ section \vill compare matches of the respondents on kinds of programs implemented. 

descriptive: (Table 56) ' . 

This table will show ESAP activities and program categories as a function of ' 
LEA descriptors such as urbtin/rura-l designation, size, percent minorltj', etc. 
This table may show relationbhips between program types and LEA demograpliic 
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characteristics. Other tables in this section will show the number of subactivities 
funded, in pl&nnlng, and implemented. These will be cross-classified' by the LEA . 
■descriptors mentioned above. , \ ' -' 

• *, 

POnCY (Tables 1 and 21) ' ' . ' ' ' 

Tabic 1 is an attempt to show if a person's expectations at the begiiuilng of the 
year are related to his interpretation of the changes that actually took place. The 
positive and negative expectations arise from the responses to the problem- solving 
questions. These Interpreted changes ^re presented as a Unction of expectations 

♦ 

by several sets of personnel descriptors, by LEA descriptors and by Intensity of 
activities as u? Table 21. This sectionals by far the largest and contains approxl- 
mately BO tables. • • ' I \ ■ 



PROJECT MAmGEMENT (Table 5i2) ' ' • ' 

Table 52 show^program inVolvement and problem solving "as a function of . , 
position, LEA percent minority, race erf supervisor and race of respondent. From 
this table we \vill be able to tell who plays the largest part in the projects, th6 extent 
to which these personnel are of different races and if problem solving-varied with ^ 
race or position. Other tables in this section will shpw us who was involved in 
planning as a function of program intensity. . ■ ^ 

^ ^ — ' — 

TECHNICAL AS^TANCE (Table 63) ' ^ " 

% ' ' ' ' 

r^^his table shows tcclinical assistance effectiv^^ss as a function of DHEW 
re^on and type of assistance. From this wc will be able to see what kinds of 
assistance arg/ lacking and what regions are providing acceptable assistance. 
Other tables in this section will determine ff the assistance received was provided 
directly with ihe^ESAP progi-am in mind., . . I ► ^ 



90 



ERIC 



A" i 



e 



o 

2;o 
o w 
P 

fr< o 

o < 

a a- 

wO 

5m CO 
O O 
w 
o 

a 

o 

5 ■ 



CO 



o 
I. 

o 
a 

o 



to ^ 

.1 -I 



o 
cs 

♦ 

o 
o 

C2 
CJ 

•n 
c 
o 



p 



XI 

I 

o 
c 

•a 

c 
o 



c 
o 

3 

^ I 
•«-> 

d 
o 
•a 

t; t: 5 

< < CO 



P 
a 



a 








d 


CI 






O 








o 








£ 




' 3 




Gi 
•D 


o 


O 


CJ 

o 




U 


u 


H 


o 








< 









CJ 



o 
c 



o 
c 

o 



3 

W 
I 

c 
o 

3 




ERIC 



91 



H 
O 
< 



O 
O 

o 



O 
C 

tr. 

2; 

;0 



c 



p 



-2 1^ 



I 



C2 



O 



P 5 



o 

.Si 1 



o 



c 
o 

I 

o 



< < 



O 



I 

s 

o 



p 

I 

CO 



rj 



o 



t % 



o 



ERJC 



92 



CO 



O 

!z; 
o 

o w 

go 
CO 



fx4 



CO CO 

<; CO 

«^ 



ft 

H 
Q 

< 
o 



< 

w 
w 

< 

< 
u 

u w 
o S 

CO P 

P 

CO 



< 






o 






>-< 

> 


< 




• 




ft 




% 


f 


6 








< 


* . * 






ft 


«• ^ 








•n ♦ 






< 


r \ 

1 






ft 








TYPE 


< 




[ON 


<4 

O. 

CO 






REG] 


ASS] 


f * ' 




j DHEW 


] TECH. 




* 



W 

w 

It -4 

H 



O 

w 
w 

> 



O 

w 

W 

w 
o 

CO 



O 

w 

w 
w 

H 



o 
w 

w 

O 



o 

H 

o 

« 



O W , 

ft c? (S' ■ , 
11 t( II 
.ft' <-o 

* -je 



ERIC 



A-8 



94 



t 

1 f 

\ 

i 

I 

4 

PERSONNEL (Table 66) ' . r 

Table 66 shows a respondent's degree and experience as a function of position, 
age, sex and race. This table shows differences due to ^ex or race dnd aee. Other 
tables will show the Incidence or classroom and non-classroom problems as a function 
of the above descriptors. . , 

. ATTITUDES (Table 57) . * 

This table will show attitudes of black and white respondents towai-d ESAP pro- 
. grams as a function of program type, whether the respondent's district was newly 
integrated or not, and the duration of the ESAP activity. This table should identify • 
racially related attitudes. ' ^ 

EXPECTATIONS (Table 58) . 

This table will sh(5w lis what was expected prior to (desegregation in and out of 
the claissroom as a function of position, experience and race. This should show us 
\rfiat problems were anticipated and we cai then compare this \yith the kinds of pro- 
grams implemented. We can also compare J;hese results with changes that have * 
taken place throughout the year. 
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INCORPOMTtD 



Dear ESAP Advisory Commiltcc Me.mber: 

- Rcsouroc Management Corporation is a private research firm under 
contract with the U.S. Office of Kducation to carry out the outside,' independent 
eyalualion of the Emergency School Assistance Progrant (ESAP) required by its 
authorizing regulations '(Part 181. 9, Title 45, Code of Federal Regulations).^ 
ESAP provided fpderal funds tTiis school year to assist eligible lo.cal school- . •> 
districts in meeting special needs arising from the elimination of racial ' 

• segregation and discrimination in their putjic schools. The purpose of our 
evaluation is to determine which type projects and acti'sjilics are most useful' 
to local districts. 

' ■ *■ ' 

• / • Your name was given by your.school distr'ict as being a member of tlie 
Bi-racial Advisory Committee for its Emergency School Assistance Program. 
Thcrelore,' it is requested that you complete the brief questions that follow 
and return them directly to- Pvi^IC irtthe' enclosed envelope. We feel it is im- 
portant to consider the activities and opinions of communiiy mcuibers such as 
yourself In our evtaluati6n but need your cooperation to provide that mform xtion. . 

• Your responses will be anonymous and your Qginie v/ill never be asso jiated 
wiU\ ony of your aiiswers. We ask for your name and address only so we will 
be able to check ok imrcturned questionnaires.. 'It is also important that you give 
your«pfi-sonal opinions, not as, you feel sonieone else would answer. We urge 
you to roinplete liiq questionnaire yourself as begt you can rather than sBt\i 
assistance from /other people in Uie school or community. 

Thank yoi4 very much for your help. Please take the neeessary few minutes, 
to eomplcte and return this form as soon ^s possible. 



Name 



/ 



SehooI'Dji'triet Name 
Youi' mailinp; address* 



\ 



/ 

100 
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^ When .did you become a member Month 
of the Biracial Advisory^, . ,^ Year 
Committee? ' ^ -I 



la. 



lb'. 



How did you become a membet 
of the Biracial Advisory 
Committee? (CHECK ONE) . > 



(QUESTI6N FOR COURT- 
APPOINTED CO.MMITTE'ES ONLY) 
Did your committee review and 
comment on the ESAP applicat3|0n . 
before it was submitted for f^ndijig? 



My name selected by an organir 

. zation to which T belong Q 
My name selected by 'School 
Board or other school 

officials . Q 

My name selected by ^jourt 

officials r Q 

'other ' ' □ 

Don't know : Q 

Y6S' □ 

□ 

Don't know □ 



2. How many times has, the 

committee met this school year? 
(CHECK ONE) ' 



^ne 

One to three 

Four to seven 

Seven times or mor6 

Don't ki)ow f 



□ 
□ 

□ 




3. • The Emergency School .Assistance Program .Advisory Committees have 
assisted school officials in variou? ways, some bf wOiich are in addifion to 
. their usual duties. Please answer the fbllowing it'ems as they apply to the 
activities of ypur Committee. - 

a. Since funding was received, has Ye's j— i 

.tlie Committee helped work out No ; q 

the plans to. actually start the ' . Don't know .A .-^ * m 

activilics? (CHECK ONE) ^ ,^^ot started yet ............. Q 

b. Since activities have started, Yes » ri 

has the Committee helped to % No , **.* j— j • 

■ solve operational problems that Don't know Q 

have come up? (CHECK ONE) Not started yet . H 

c. Has the Committee been asked Yes Q 

to advise school 'Officials on 'No Q 

ESAP policy matters? Don't Imow ** Q 

(CHECK ONE) Not started yet iti"! ] □ 



4. Overall, do you feel that the . Yes \ □ 

educational authorities in yoUr No .-"^^..j O 



district are responsive to the 
committee's wprk?- (CHECK 0~NE) 



Don't loiow .\. ..\. , □ 



To what extent do you feel involved Very involved .} '* □ 

in the Emergency School Assistance Somewhat involved □ 

Program. in your district ? (CHECK » A little involved .* .' .' '. .' .' .* .* □ 

2M) Not at all involved .' .* .* .' .' .* .' .* . .^ □ 

4 



5a. ^Ilas the extent to which you have Just as expected □ 

. been involved in the Emergency Less than expected f^! □ 

School@*sistancc Prograjn in your More than expected □ 

district tunicd out to be just as you - ' * 

.jBxpeclcd, less lliM you expected, ^ 
or more than you expected? ^ 
(CHECK ONE). 




6. To your Icnowledge, did the school 
ydistrict make public the names of 
advisory committee members ? 



Yes' • O 

No ....t...^... □ • 

Don't laiow 

- / 

Yes V /. ' 'q . 

No i • □ ? 

Don't know \ \ Q 



7. Have you observed or participated 
if! ESAP project activities in' your 
fchool district? 



8.^ How true is each of the following statements fo2> the Emergency School 
Assistance Ptogram in your school district? \ 



^ a. It is a program of specific 
I activiti(^s designed to help 
minoriti students. 

r b. It is a program to .tfiake 
desegregation easier 
in the^chools. V 

c. . It is a program to give 
genern^ aid to help pay 
for all s&hool costs. • 



Sonie- A Not 
'Very what yLUtle at all Don't 
True True/ Time • True Know 




p 



□ 



□ 



^ — : , •__ 

$1,000, 000 or more ..; 'Q 

$500,000 - $999,999 .' " □ 

$200,000 - $499,999 .'k □ 

$100, 000 - $199,999 Q 

,$50, 000 V $99,999 Q 

$25,000 - $49,t)99 □ 

$0 - $24,999'.. .; 

No graiil Q 

* Don't laiow [T] " 

• ' r 



What is the size of the Emergency 
School Assistance Program grant, 
in your school'district? '(CHECK 
ONE) . 
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10, As far you IcrtcHv, which of the follo^^ing list of Emergency School ^ 
Assfstance Program activities are trUvinjT place in your school district? 
. (aiECK ALL THAT APPLY) ' ^ 



Special Community 

Programs Q 

Counseling . . . ^ • . • • Q 

Special Curriculum 

Revision . . . /. O 

Teacher Traming ^ □ 

Teaclicr Aides « . . : □ 

Studcnt-to-Stoident 

Programs . . . . ^ . . D 

Bussing ; . • D 



^Compens^atory or Remedial: 

Education Classes Q 

Comprehensive PlaniTing . • . . • Q 
Remodeling or 

Renovation . 

Portable Classrooms Q ' 

Other Personnel • . . . . 5/. • 'Q 

Other Materials ^ Q 

None ♦./.•^ \ Q 



!!♦ Do you feel that there has been a change for the better or worse or no^ 

change at all in your school district this school year in the following areas? 

.... ' Don't 





• 


Better 


Worse 


No Chnns^e 


Know 


a. 


Relations behVeen students of 












different races . , , , 


□ 


□ 


D 


□ 


b. 


School-commujnty relations .... 


□ 


< □ 


□ 


■ □ 




Relarlioiis between faculty and ^ 










students of different races . . . 


□ 


D 








Classroom performance of 












black students . . . 


□ 


□ 


. n 






Classroom 'Performance of 












white students ; 


□ 


□ 







PLEASE. PJUNT OR TYPE YOUR RESPONSES TO THE NEXT TIlHi:)' QUESTIONS. 



T 



12. For Up areas in which a clThngc for'llic better was noted aboA-c, dtscrive. 
what you feel caused Iho change^ (Ex'PLAJN) ' ^ 





\ 
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13. For the areas in which a change for ihe worse was noted, describe what 
. y-ou feel caused Ihp change. (EXPLAIN) / 

I 




14, All in all, do you think your 
• school district\didW good job 
in planning its ESA^p application? 
(CHECK ONE) , 



l^a. Do you thinlc your school • 
•district did a good*'joD in 
planning for carrying but • 
an ESAP program? \ (CHECK) 
ONE) i ' 




. . - -- P 

No\),..... ■ □ 

Don't jpow /. . .. Q 

No .started yet O 



14.b. Do yxin thinli your schooll 
,^ • district ,^id a good job in \ctually 
\ carrying out its ESAP program? 

\ (ai-Ea<: ONE) 



Yes □ 

No ....I □ 

Don't knckv " □ 

Not startd^d yet *. . ; jO 



PLEAsAfILL IN THE FOLLOMNG INFORMATION ABOUT YOURSELF 



' 15. AGE: . Under 2U D 21 - 25 □ 26 - 30 □ 31 - 35 □ 

^/ 36 - 40 □ 41 - 45 □ 46 - 50 □ 51 or older Q 

^ETHNIC GirOUP; Black □ White □ ' , Spanish Speaking 

17. Do you have nny cluldrcn currently . Yes Q 

attending- a public schopl in your ^ No ... , V * * pi 



i 



THANK YOU VERY I\IUCII 
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